Jekyll2020-03-26T22:01:50+00:00https://sagnikb.github.io//feed.xmlThoughts, QuantizedHomepage for Sagnik Bhattacharya, Electrical Engineering PhD Student at UMDSagnik Bhattacharyasagnikb@umd.eduIsolation on the shoulders of giants2020-03-26T14:00:00+00:002020-03-26T14:00:00+00:00https://sagnikb.github.io//blogposts/isolation-giants<p>Scratch… scratch… scratch…</p>
<p>The sound of the quill pen dragging across the paper was like music to him, drowning out the mooing of the cows. The light rain throughout the day meant that he hadn’t needed to take them out of their sheds. He was thankful for that; he didn’t have to run after those cows and get feed for them. He had celebrated by spending his time in his room, thinking and writing.</p>
<p>Such days had been few and far between ever since the university had shut down a couple of months previously. As he dipped his pen into the ink bottle, he glanced out of the window. He had wanted a day when he could work uninterrupted, but now he yearned to go out into the yard. The day had been long, and he had not made much progress. The numbers shuffling around in his head just didn’t arrange themselves in a way that gave him any insight. He looked a little closer at the sky - maybe it did look a little brighter? It would need to clear soon if he wanted to take his walk outside, otherwise it would be dark and it wouldn’t be quite as refreshing.</p>
<p>He wrote a few more lines, but the glance out of the window had diverted his attention permanently. Stretching back in his chair, he looked at what he had written. He had slowly understood some of the principles that make the world run, walking further on the path indicated by Galileo than Galileo himself had. It had not been easy - he had found the math that he had at his disposal to be much too clunky to describe the beauty of nature’s inner workings, and he had developed his own ways to describe how nature itself moves. He often thought of the world around him as some kind of clockwork, much like the pendulum clock at the university, except that a few simple gears weren’t the only things he could explain - his ideas could do much more than that if they were true.</p>
<p><em>If</em> they were true? <em>Of course</em> there were true! Three small ideas were enough to describe every motion around him. If the sheer elegance wasn’t enough, they were <em>powerful</em> too!</p>
<p>He was still not satisfied though. He had imagined the clockwork nature of the pendulum extending to many things around him and it had worked beautifully. Especially on a farm, he could see it in action everywhere - pebbles rolling, people and animals walking, someone raising water from the well - his ideas were everywhere!</p>
<p>He was itching to see if they extend even further. Could this clockwork structure extend further, to the moon, say? The sun? The stars?</p>
<p>He returned to what he was writing. His thoughts placed him among the stars - what chance did a measly yard stand against the universe?</p>
<p>There was no clock on the farm. It disappointed him, even though he had several sundials, all of which he had made himself, strewn all over the farm. They weren’t so useful on a cloudy, rainy day.</p>
<p>He liked looking at clocks. Especially the precise craftsmanship of the pendulum clocks, and how they were independent of the weather. He had thought of making one - but he had been keeping his craftsmanship to one side. The mathematics was too interesting at the moment.</p>
<p>A sudden brightening attracted his attention back to the yard - the rain had stopped, and the clouds were clearing a little! He quickly set aside his pen and rolled up his parchment and within a few moments was out of the door.</p>
<p>He could tell that the sun had just set even though the clouds hid almost everything. A few stars could be seen in the region of the sky that was devoid of clouds. He felt strangely close to the stars - he thought about them so much, after all. On second thoughts, though, one of them seemed to be a planet. It wasn’t twinkling. Venus, maybe? <em>The evening star</em>.</p>
<p>He felt the wet grass below his feet as he walked to the far corner of the farm. He would sit underneath the trees there and have a good hard think. That should open his mind up.</p>
<p>He reached his preferred spot. The branches of the tree seemed laden with fruit. He considered plucking one, but decided against it. It wasn’t time to eat yet.</p>
<p>He could feel becoming one with the nature he wished so dearly to understand. The plan was to stay there and wait for inspiration to come, the way it had come several times before. He shut his eyes, and just then…</p>
<p><em>Thud.</em></p>
<p>A sudden sharp blow to the head broke the spell. As the apple fell to the ground with another thunk, he could almost feel his spirit drop like it had done. Why did it have to fall <em>just then</em> and spoil his thoughts?</p>
<p>Why did it <em>have</em> to fall?</p>
<p>Why <em>did</em> it fall?</p>
<p>He looked at the apple. And then back at the stars in the sky. The moon showed itself for a moment through a crack in the clouds. The clouds were clearing further… and not just in the sky. He could feel realisation and discovery send a shiver down every part of his body.</p>
<p>The numbers were in order now.</p>
<p>He couldn’t even be angry at the voice that came from far away.</p>
<p>“Isaac? Where are you, Isaac? Go right now and see whether the cows have enough food. You should have done that an hour back. Isaac?!”</p>
<p>He took one last look at the apple, before walking slowly towards the cowshed.</p>
<p>He had his answer. The clockwork did extend further.</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/SFuz9guq24cTnNM0Y_3YaJAkr_SRSw5R6Vi1iicQBV5f0DBs3NV969DoX5zhZL-3k_-dKKZeZrAPVvF1CTiDHtyMLFquMh8kGd_sM1i4SXjPCDQTNxdZOJ9BWxhfKMtkg1vU6QOfA0o=w2400" alt="apple" />
</figure>Sagnik Bhattacharyasagnikb@umd.eduAnother person was once isolated. This is a day in his life.To the bit of my heart I left in Hong Kong2020-01-15T16:30:00+00:002020-01-15T16:30:00+00:00https://sagnikb.github.io//blogposts/Hong_Kong<p>I remember the car journey from the airport to the centre of the city. By the end of the first day, I had a crick in my shoulder, from instinctively looking up to find the tops of buildings I could not see. I was coming from Kolkata, after all - a much shorter and fatter city.</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/vH1aquBlny3LHGm4ZnorGC-gD3geUMXyUHyP5p_ep_v203Tp3OKFAYfa4k4foHU5beSMUYXtfhlzF_7kR1PASUuCsPPWzsdx3cD_42ryKGNU1zH_we42LmJxJ6KEoF8_hrRlN5y_mEw" alt="View towards Victoria Harbour from Tsim Tsha Tsui" />
<figcaption>
<strong>View towards Victoria Harbour from Tsim Tsha Tsui</strong> Tall buildings dressed in glass. One side of Hong Kong.
</figcaption></figure>
<p>I remember another ride, on a bus this time, to the Tai O fishing village, and feeling transported to a different world, far away from the tall buildings dressed in glass. Here there were villages and shops selling sea-food that I had never seen before. Was this really less than an hour away from the city? I would later discover that the tall buildings were just one facet of the place - about 70% of the region is forested land!</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/_fWTZK5cMnlLUN4IjeUvdewSyQNw4mpioYp0SMmqIyht1GlsZ3oXBVXy14FM0QX6rAtDZCLAcErQkl0IfKzT2i2hhQ04Eib11WBz4xGE158dtKzqTSWODiQtJRekAZ0IUPV58aZW4ZI" alt="Seafood store at the Tai O Fishing Village" />
<figcaption>
<strong>Seafood store at the Tai O Fishing Village</strong> featuring inflated pufferfish above the door.
</figcaption></figure>
<p>I remember the first time I went to the CUHK campus, at first loving the fact that I would be spending one-and-a-half months in that green campus on that hill. Almost immediately, though, I remember loathing the fact that I had to trudge up the same hill, dragging a heavy suitcase on steep footpaths on a warm and sweaty summer day. It reminded me of Kolkata, the heat and the sweat.</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/rZIT3R_Oy2jpyBOOjIQLLVe-weUDNmAUcQb1NwMi9rT2Z88QM4VXUoEDyWFx_PepSTTxkdDxFDXnlSDn4XYp1JH5SSIUTu2zFuegiW53qRGScB7xKNZyhgq-3vH2GstTpdk0Ak0H65I" alt="The Ho-Sin Hang Engineering Building" />
<figcaption>
<strong>The Ho Sin-Hang Engineering Building</strong> that houses the Institute of Network Coding, my workplace for the summer.
</figcaption></figure>
<p>I also remember how I was looking at a map on the street corner, trying to figure out where the room I was going to stay in was, and the kind elderly lady who appeared, asking me what I was looking for. She then walked with me, in the opposite direction to where she was originally going, into random buildings and even more random elevators, until she put me on the road to iHouse, block 3, and ensuring that I could get back to the place where she had found me. We talked on the way. She said that people new to the campus often get lost. I would learn over time.</p>
<p>In a couple of weeks, I could walk up from the metro station at the bottom of the hill to iHouse, where I lived, and from there to the Ho Sin-Hang Engineering Building, where I worked, taking all the possible short-cuts through buildings and the elevators there-in and not even being out of breath. The campus had adopted me. Revealed to me her secrets. The elderly lady on the first day had made me feel <em>like</em> home; this made the place feel home.</p>
<p>I remember going to one of the many canteens on campus on one of those first few days, walking past the densely forested slopes, accompanied by my parents. I remember looking at a sign that said that the canteen was for students and employees of the university only. I remember asking the person at the counter whether my parents could also come in, but her not understanding English beyond whatever was on the menu, and the man who walked up to me, asked me what the issue was, and when he heard me, who gave me a big smile and said ‘But of course everyone is welcome!’ and then proceeded to explain to me all the meal plans that I could avail at that place. </p>
<p>I remember exploring different kinds of food at the many canteens on campus, trying real Chinese cuisine for the first time. You know I grew up in Kolkata, and my liking for chowmein and chilli chicken or fried rice and chicken manchurian. I had heard that Chinese food in China is nothing like Indian Chinese. Very true, that. What I had also heard was that if you like Indian Chinese, you won’t like Chinese Chinese. I completely disagree! You know how I loved every single dish I had there, including a very hot chicken dish that came with a free litre (!) of Coca Cola to make it bearable. I also learned to use chopsticks, so much so that by the end of my stay, having chowmein was easier using chopsticks as compared to a spoon and fork.</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/s1ECkBs4oB65oc-OBNnKY3ruyT-aRPRUfMxxKD9QuP0uuLnDcNwB4zev70uaF_Z-HxsNQWL1Q6-_XHoxQqZGZVzeEBLB-JjsNtDxzb5GmpGlL4zaCzyEvlVv2MVuw31u2UcD-ZnqWMc" alt="Spicy chicken" />
<figcaption>
<strong>Spicy chicken</strong> with a litre of Coca Cola on the side. I absolutely needed the Coca Cola!
</figcaption></figure>
<p>You also know how eagerly I waited for mooncakes to hit the shelves as I was approaching my last few days in the city, and how fulfilled I felt after buying the two cases as they came to the stores just on the day before I left.</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/j7-UhUf7NcUi9pM-LFS6cvFqkvPFMvrRRI6INZ34XOdCRAUM0CtTVP5tSSNBZWwwSJ-YsNy5sX-RgghCin2gXID_lKCNUSGyOvhS6K6ZaT4QF85pfV3Gr2bvTF1Qe6DAsKtkSrJ8wvs" alt="Mooncakes" />
<figcaption>
<strong>Mooncakes!</strong> The mooncakes came in this beautiful box. Kee Wah had a store at the airport stuffed to capacity with them.
</figcaption></figure>
<p>I remember being naïve and thinking I could ‘pick up’ some Cantonese while staying there, and my disappointment at discovering how different and therefore complicated the language is. You know how I found out about name seals, and how on one of my weekend trips I sought out the small, telephone-booth like shops along <a href="https://www.scmp.com/article/730471/chop-alley">Man Wa Lane, Sheung Wan</a> that sold them, getting my name translated and getting one made for myself.</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/Bz7_QTX99pFeCAuW1uYlOEXZKus5pxUwOgOcF7uHmnx0PrZq8eqPaqhKlpLz9o9HYWkICymnJ81Tmhwa1NP1b2hOxMsqVvhSPkO9GqyCW3Aucd4mM0iMqyvCMUxp_XPiyAIIfwDpL3A" alt="Name Seals" />
<figcaption>
<strong>Name seals</strong> On the left, my name seal and a porcelain ink pad containing a vegetable dye. On the right, the nice old gentleman who made it for me. His fast and precise movements were a joy to watch.
</figcaption></figure>
<p>I remember how, after my parents left after us enjoying the city for a week and also celebrating my mother’s birthday, I left every weekend to explore the city. It was the first city I really roamed around alone. Every weekend I left iHouse, walked down the hill to the metro and travelled to the middle of the city, seeking out something I had read about the day before on Google and found interesting. While coming back my phone would be almost out of charge, and I would need to know which metro stop to get down at and the way back to my room because Google couldn’t help me any more. I can now take my phone and travel fearlessly, cheaply and conveniently to any part of a modern city, having since done so in both Paris and Washington DC.</p>
<p>I somehow also love how I have recounted so many happy memories without even mentioning the research group I was part of, the amazing advisor I had there and what my time in the city means to me academically. You know how I had only recently changed my research area from quantum computation to classical information theory, and how stressed I was about the switch. You know how very happy I was when I finally had an academic internship for the summer of 2018. The time in Hong Kong not only gave me a wonderful advisor whose recommendation letter and general guidance made navigating graduate school applications and finally getting and deciding on an offer a much smoother experience but the papers I wrote directly based on the work I did there helped my applications a lot. You know how I used to be iffy about embarking on a PhD, but how one conversation with Sid gave me the confidence that I wouldn’t feel like a fish out of water while doing my PhD. We went on research meetings on the beaches, enjoyed Chinese food at restaurants recommended by my advisor there and so on - there’s a lot!</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/E4qWxGOJVDN6Z-JGIP8NYswJoaVAZTTj0JL3whBe_bF3mY7NxVs7RaDsOeSQiIuZP0O5MnU3QO1V0sw6XIkl2gS-NlsS500xIpm04EKqv18KK6Tl3LwpoVk7d658xHR2r0ImUcggvLo" alt="Research meeting at the beach" />
<figcaption>
<strong>Be-search</strong> We had research meetings at the beach, which we ended up calling Be-search. This photo was taken at the Cheung Chao beach. My be-search talk on zero-error information theory and the Lovasz theta function was in progress when this large labrador decided it was an interesting topic. When we stopped to watch him, he moved on to the football, which didn’t last long.
</figcaption></figure>
<figure class="">
<img src="https://lh3.googleusercontent.com/RUs-rj-U7jeFKb4FCKHofBCYrJm9fuYm9_LNLOV_tNJ0GaXtaGR7JGx8K_unHLtqR2BMEBt3TYIqh0AtmY_DIcT0wODee3KiMi6F5L9b3j1ofJDq6tvCzSnXb7yiDDmhMA_lnTepoiM" alt="Institute of Network Coding" />
<figcaption>
<strong>Institute of Network Coding</strong> featuring my weird smile.
</figcaption></figure>
<p>I grew up travelling to and from school on the Kolkata metro hearing frequent comments about how I should carry my bag differently, and even how schoolchildren with heavy bags on a crowded metro were an inconvenience. I can now contrast that with the day I left the city, carrying two heavy suitcases on the metro during rush hour, half expecting someone to tell me I couldn’t carry them, or to berate me for doing so. I don’t know whether I didn’t overhear any stray comments simply because I don’t understand the language, but you know what, I will go out on a limb and say that there <em>were</em> none. It seems in line with my other experiences in the city. It felt like a pat on the back, a last bit of understanding, a last memory to cherish.</p>
<p>I now read about the turmoil the city is going through for months now, and I remember all of these memories. I see protesters. I see university campuses under siege. I see pictures of roads that I know blocked and tear gas shells flying over fields that I have walked past. I feel like people I know are in the crowds of people I see on the news. Maybe they are. When I also see videos about people recording goodbye messages to their loved ones, not knowing what the government’s reaction might be, I feel scared. But some part of me also feels proud that some people somewhere in the world are standing up for their rights. I feel concern for the well-being of the friends I made in my few weeks in the city, and the places and institutions I got to know. It feels like another lesson from the city, even though I am now miles away - making me realise, for the first time in my life, that the crowds protesting are made of <em>people</em>, and are not just faraway masses in faraway places.</p>
<p>I hope you are well. I hope the city resolves its problems and remains as welcoming as it was to me when I washed up on its shore for the first time. I hope that I can recognise the city when I get an opportunity to go back there and reconnect with you.</p>
<p>You, the bit of my heart that I lost on the streets of Hong Kong.</p>Sagnik Bhattacharyasagnikb@umd.eduMy stay in the city, more than a year onThe Four Year Transform2019-12-31T14:30:00+00:002019-12-31T14:30:00+00:00https://sagnikb.github.io//blogposts/Fourier<blockquote>
<p><strong>Quick note</strong> - This <a href="http://voxiitk.com/as-we-leave-9/">first appeared</a> on Vox Populi, the campus journalism body of IIT Kanpur. They deserve credit for getting me to beat my procrastination and write what follows (no mean feat), and for editing this and stopping me from rambling. In particular, credit for that last bit goes to Ayush Agarwal.</p>
</blockquote>
<p>I remember the first time I went to the PK Kelkar Library and looked at the vast collection of books, and the goosebumps I had when I first roamed about in the Physics section. It was a few days into my first year, and when I called home later that night, I excitedly declared to my mother that if I read one of those many, many books per month, I’ll have learned a lot before it was time to graduate. Needless to say, like most of the plans I had made in the first few weeks that I had spent on campus, this did not work out. It did not work out to such an extent that the only time in the past year that I have stepped into the library (apart from graduation formalities) has been to get print-outs and one other time to roam in the physics section, trying to recapture the naïve awe from my first year. What happened in between is something I will call the <strong>Four-year transform</strong> and will try and spend the rest of this essay trying to elucidate the properties of this strange beast.</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/QgZk09wFUPohilLW1ih0wC7YYl45p4ziQEJdXYmsNohi9zjn1dBPpLwOVEotdyWZIdGN9awWofCGcal8j2jJABDrE2ODyAdo8FaZCdXlSgPKxFI7Flh5wic1xSCVoqz6kPn3lP-f7ps=w2400" alt="ENG423 class photo" />
<figcaption>
<strong>ENG423 class treat</strong> At the end of my personal Four-year transform. Current Issues in Linguistics, one of my favourite courses with one of my favourite professors, and a stellar group of fellow students.
</figcaption></figure>
<p>Here’s how the Four-year transform operates.</p>
<p>On entering the campus, we are faced with a multitude of choices about everything related to campus life. Some will probably not influence much of our future lives, like the question of what to do with the roll of TA101 sheets after the course ends. Others do have <em>some</em> ramifications later on. Questions in the early days like, “Which clubs to join?” and “Which Takneek competitions to participate in?” later change to, “Which courses to take?” and “Which professors to approach for projects?” It is clear that the choices in the second category are not scalar quantities – for example, if one chooses to join the Dance Club and also becomes active in Science Coffeehouse, it’s hard to imagine how they can both directly contribute to the same outcome. Therefore, we can say that the effects of these decisions add like vectors and that it’s only the resultant of the different pulls that matters. What the <em>Four-year transform</em> does is that it changes the magnitude of some of the vectors and rotates some, both of which change the resultant. So to understand the resultant, we need to understand the properties of the Four-year transform.</p>
<p>Now that we have some idea of what the transform is, it is instructive to try and see what it <em>doesn’t</em> transform, namely, its fixed points. For me, it didn’t change my love for science or the motivation to do research. It has matured, sure, and I certainly understand what research involves far better than I did then, but the essence of my fondness remained the same. It should be a valuable exercise to try and figure out what your fixed points are, or, what you want them to be because it can reveal what is most important to you. I am sure everyone has <em>some</em> fixed point(s), even though the specifics may be highly variable from person to person.</p>
<p>Coming to the things that it <em>does</em> transform, we can try and find some properties of the transformation. Briefly, they are</p>
<ol>
<li>
<p><strong>Integration property</strong> – The only way to judge the effect of the transform is to observe the person at the end of the four years.</p>
</li>
<li>
<p><strong>Time-scaling</strong> – The amount of time spent in a particular activity has some effect on the future impact of that activity.</p>
</li>
<li>
<p><strong>Strong non-linearity</strong> – More effort doesn’t always lead to proportionately more payoff, and</p>
</li>
<li>
<p><strong>Time-shifting</strong> – Time shifting of events changes outcomes in unpredictable ways.</p>
</li>
</ol>
<figure class="">
<img src="https://lh3.googleusercontent.com/itn3aj6F60U94tRJsHwt2OMQtoUQwXRLYbSu2uIbqDis3LfP-1L8Epk9_p0vL6FFNiqpwT3QtCCNUW0jcdZfjyoJ_YbhowKP5iY68oAVCkFTgzVu-oTn66qff-lF_hfcJxfR6hcjKIA=w2400" alt="Science Coffeehouse" />
<figcaption>
<strong>Three generations of Science Coffeehouse leaders</strong> I decided at the end of my first year that Science Coffeehouse is the most fun group on campus that I am part of. I still stand by that.
</figcaption></figure>
<p>Now, let’s try and tackle these one by one, breaking down the aspects and its effects, trying to build a parallel with real world scenarios.</p>
<p>The integration property tries to capture the differences between the way your life in IITK will be interpreted by you yourself on one side and the rest of the world on the other. It is very unlikely that anyone will care about the specifics of your education at IIT Kanpur after you graduate. All that you study in the courses, all that you learn in the clubs will only leave its mark in what you know and can apply. It is too early for me to say whether your CPI will matter in the long run, but even if it does, it is likely that people will care about only the number, not how you got it. In some sense, what you do on campus will be represented in life as a few lines or some number on your resume – no one will be interested in how they got there. For example, my fifth semester was by far my busiest and craziest semester, and at the end of it, I got two of the grades that I am personally most proud of getting. One of them was a B in EE370, Digital Electronics. Due to circumstances, I was not able to study for it at all, and my preparation for the midsem and endsem consisted of watching the lecture videos that Prof Mazhari so kindly sent us, three hours before the respective exams. I think that, given the situation, I have grounds to be happy about that grade. The other one was a B in PHY690G, which was a course on coherence and quantum entanglement meant for advanced Physics undergraduates, MSc and PhD students before they embarked on real research. I am rather proud of the fact that I, as a third year UG in a different department, was able to do enough in that course to get a B. But, both of these will be one among many B’s (among many A’s and a sprinkling of C’s) on my transcript. Given the fact that many PhD applications are judged in around 5 minutes, I don’t think anyone even noticed the B’s, let alone my personal story behind them.</p>
<p>In contrast, everything that you do on campus will leave its imprint on you, because your own personal story is the sum total of all of these little things. The wide world might not care about the specifics of the story, but you do – for example, the two courses above taught me how to take on challenges that might have felt insurmountable, what to focus on while having to learn something quickly and so on, skills that came in useful later in life. The balance, therefore, must be between optimising both what other people will care about and what will be important to you as a person. And, as usual, the balance is hard to reach.</p>
<p>The best illustration of time scaling comes from the following. There are two ways to approach a course and its exams – either attend most of the classes, do assignments on time and go to the exam well prepared or mug up three days before the exam and trust your brilliance (if any). The difference is that in the latter case, you have compressed a semester’s worth of studying into three days, and that means that the impact that the course will leave on your knowledge, long term, will be scaled down by an appropriate factor. The more time you spend on a particular topic, the more comfortable you’ll be with it (and the more the long term impact) – importantly, and vice versa. Take a wild guess what the long term impact of EE370 (above) on me was.</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/cx1g8bohXmoueIv7qxKMT8lItPTQ5_yb4T0_KcWat0xJO1up2QDbcw6L5TmCSAg02Z7FqSMSQttqVq6xfFFOSUSMWiFjqFwrv_ny6GNGf_RUhRs3d_yAA4AvjTTnae5DnEBzbSZhGuw=w2400" alt="EE Formal Photoshoot" />
<figcaption>
<strong>Electrical Engineers</strong> Some people who made Electrical Engineering special for me.
</figcaption></figure>
<p>Before moving on to the last two properties, let me narrate another small story. The first seeds of me studying information theory were sown in the middle of my second year during a series of talks organised to celebrate the birth centenary of Claude Shannon, the father of information theory, but, before the third year summer when I went to the Chinese University of Hong Kong (CUHK) for a research internship in information theory, I had only studied the subject for a cumulative time of around a month. At that point, by comparison, I had studied physics for more than two and a half years – so much so that I was able to handle the PHY690G course mentioned above. Fast forward a year, and I have a couple of publications in information theory and in a few months, I will be starting a PhD in the same area. I stopped doing Physics about a year back.</p>
<p>The points to note are the following. One, the timing of Shannon Centenary Day was crucial – a few months earlier, and all the talks would have gone right over my head. A few months later, and I would likely have already chosen another topic to work on. Either way, my life could have been very different. The impact of an event, therefore, depends crucially on when and how it occurs – which is the time-shifting property. Also, the pay-off per unit time spent in physics and information theory were very, very different for me – which is the non-linearity. The irritating bit is that the only way to judge the impact or pay-off of a decision is with the benefit of hindsight, the importance of which becomes apparent in the next paragraph.</p>
<p>The non-linearity and the time-shifting property together imply that it is difficult to predict the eventual impact of your choices and actions. This lack of control and predictability is in sharp contrast to the life in the JEE days, where the relationship between effort and outcome was much less messy. The following, I feel, is the most important thing that I learned during my own Four-year transform: Most of the time, the decisions that you must take cannot be fully justified at the point of time at which they must be taken and each of these must then constitute leaps of faith, big or small. The most effective way to handle this randomness, I feel, is to try and take the best possible decision given the data available to you at that point of time, along with the willingness to take responsibility for the results and the ability to handle the outcome, good or bad. If the outcome isn’t good, at least you’ll have the satisfaction of knowing that at every point leading up to the outcome, you took what appeared to be the best available path.</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/rcbL1ZjNTHEYZZXACEZLbKMsjjio52OLQG3h0LLj3Hn1_r7gSBg03CqH0JQX-crGbP4PLDh0XqoYnsbjkVbbvPYd3NglIIFML4gDK3J7rCz_mhODUjy9_-2JkbIrZZ5loYGL1pQSBkI=w2400" alt="Saraswati Puja" />
<figcaption>
<strong>Saraswati Puja, IITK version</strong> Sarastwati Puja had been an important day of the year throughout my childhood, and I IITK didn’t let me miss that.
</figcaption></figure>
<p>The next natural question, therefore, is that given the unpredictability, how to decide what the best choice is at a given point of time. To answer this, I must invoke the keen insight of Prannay (one of my closest friends and in general a person worth listening to) and point the reader to the birthday paradox. Briefly, the birthday paradox is the following (perhaps non-intuitive) statement – in a group of 23 people, there’s a 50% chance that two people have the same birthday. The idea is that the probability that none out of several low probability events happen decreases pretty quickly when the number of events increases. Therefore, it would perhaps help to choose, early on, a target and a sufficiently large neighbourhood set of outcomes that you want to reach when you graduate, and then work hard enough and smartly enough such that your probability of achieving something in that neighbourhood increases. With a smart choice of neighbourhood (such that a sufficient number of events exist in it) and an honest assessment of your own skills (which will tell you the probability that you will achieve something in that set of events), you can try and maximise the chances that you’ll be satisfied with where you are when you graduate.</p>
<p>We only discussed the birthday paradox in this context when we were both deep into our final years, so I have no idea whether this works. But, it does share some characteristics of a good theory. First of all, it’s elegant. Second, it does explain some things well. For example, someone asked me how much CPI matters for an MS/PhD offer. I said that a higher CPI doesn’t hurt, and if you can build a great research profile and also maintain a high CPI, then go ahead and do it, also adding that <em>no-one knows the precise trade-off</em>. This could be interpreted as an aspect of increasing the probability of the target outcomes. The theory also explains the fact that those with very strong research/job backgrounds or very high CPI’s or both get good offers for higher studies/jobs – essentially they make the probability of some of the outcomes so very high that the chances are low that they’ll miss that. If you are in such a utopia, great! If not, invoke the birthday paradox to come to your rescue.</p>
<p>Well, that’s my two cents. Maybe some of you would be kind enough to apply this in your lives and tell me what the outcome is so that there is some data to judge its efficacy. Good luck, and may the Four-year transform be with you!</p>Sagnik Bhattacharyasagnikb@umd.eduAn essay with random fundae on college life packaged in a giant punFrench Duels2019-11-28T00:00:00+00:002019-11-28T00:00:00+00:00https://sagnikb.github.io//blogposts/French_Duels<p>There was a man sitting at the desk in front of me. It was night and the room was dark, a darkness only broken by a ray of moonlight that found its way through the open window and a candle that was lighting whatever the man was doing. A glance outside said that I was in a room overlooking a deserted street in a city that was in deep sleep. Other than the occasional cry of some bird and some far away drunken song that sometimes wafted in through the window, there was scarcely a sound other than the sleepless scratching of pen on paper emanating from the man hard at work in front of me.</p>
<p>I walked up until I was standing just behind him and cleared my throat in an attempt to attract his attention. He didn’t do so much as glance at me! On looking at what he was doing, I failed to understand a word for it was definitely not any language I knew, even though the script was the Latin script. I could rule out English and German for sure, but there were too many other options. As he continued to write, I was a little surprised to see some known algebraic symbols. Even though the notation was archaic, it was something related to solutions of some equations that I had seen in my college math classes! This man was clearly a mathematician with what I thought were impressive powers of concentration, for even after I cleared my throat more loudly, I received no response.</p>
<p>“Excuse me?” I said in English, loudly enough for him to hear, but still there was no reaction. I tried to poke him, or push his chair, or pick up one of the many papers, but my fingers seemed to pass right through everything. I could move nothing, change nothing, and he took no notice of me. Wherever I was, it appeared as though I couldn’t interact at all with physical objects in this world, and from what I could see, no noise I made reached the man in front of me.</p>
<p>“Don’t try to attract his attention, you won’t be able to do it!”</p>
<p>My initial shock was quickly replaced with relief, for the voice was of my guide! She had been standing very close to where I had originally been, but the long shadows had hidden her too well. I could still only see her silhouette and she made no attempt to move into the light.</p>
<p>“Welcome to Paris,” she continued, “You seem disoriented. That’s natural, happens to everyone the first time they experience it, though I must say you are taking it rather well! Until you remember how you ended up here, let me fill you in. This is the year 1832, and it is the month of May.”</p>
<p>Even after everything that I had seen or heard, I still couldn’t believe that this was possible. But what choice did I have?</p>
<p>I started putting two and two together. We were in Paris and there was a young mathematician probing the secrets of algebra. It really wasn’t very difficult.</p>
<p>“So that… that’s Galois?”</p>
<p>“Yes, that’s him.”</p>
<p>“We can’t affect anything in this world right? I tried to move stuff, but my hand seemed to pass through solid bodies! He doesn’t even seem to know we’re here!”</p>
<p>“We can’t change events in this world, we can only observe. if you travel back in time, you cannot change anything that happened. You can’t interact with objects or people and no one will hear any sound you make, except other time-travellers like yourself. Imagine everything here is a hologram, you will just pass right through any object.”</p>
<p>“That’s exactly Stephen Hawking’s chronology protection conjecture, right?” I asked, while observing my leg which seemed to appear directly out of a table. It was a strange feeling! I hadn’t even noticed that there was a table where I was standing. The question was of course rhetorical. I had only recently read about that conjecture. My companion must have known this, for she didn’t answer.</p>
<p>Before I could ask any of the many questions in my mind, we were interrupted by a groan from Galois, as the candle went out. He quickly got up and relit the candle. There was a picture of a woman on his desk that I hadn’t noticed before, and he seemed very upset that in the darkness he had caused the picture to fall on its face. After correcting it almost reverentially, he gathered some sheets together and continued to work. He seemed to grow more restless as time went by, repeatedly glancing at the clock that hung on the wall. It was four in the morning.</p>
<p>“You can go closer, like you did before. He can’t sense your presence, remember?”</p>
<p>I did. There were sheets of paper all over the floor of the room, every piece of paper filled from corner to corner with mathematical symbols only some of which I could vaguely recognise. There was a smaller desk just beside the desk on which Galois continued to work, and on it was something that seemed strangely out of place in what was definitely the abode of a young mathematician – an old fashioned gun, recently oiled and ready for use. It was now that I fully understood what I was about to witness in a few hours and why I had been brought to this particular day in the life of a mathematician I admire so much.</p>
<p>My thoughts were interrupted by the sound of breaking glass. Somehow a bottle of ink had been overturned and was now lying in pieces on the floor, and there was ink all over some of the sheets. Galois looked close to tears but that only stayed for a moment before his steely determination came back. After quickly cleaning the mess and opening a new ink bottle, he started writing again. I could now totally understand his urgency and a feeling of dread was spreading slowly throughout my heart. I wished I had never come here, for it would be difficult to witness what I would surely witness.</p>
<p>I decided to explore the room a little bit more. Everything around it had an air of neglect which is difficult to describe. It was as if the room was frequently uninhabited. There was stale food in unlikely places. The books, shelves and papers in the room were all almost uniformly dusty. I couldn’t pick anything up, of course, which was a shame, for there must have been documents there that could’ve given more of an insight into the great mind which I could see at work in front of me.</p>
<p>My guide had been entirely silent for the entire time I was roaming around the room. Now I heard movement from her side. She was looking out of the window and from the amount of light it appeared as though dawn was approaching. For the first time I could see her in some sort of illumination and she looked remarkably human, except that there was no hair on her head and she had larger than usual eyes which she had slowly turned towards me. Not for the first time, I felt like I was being X-rayed.</p>
<p>“You are surprised to see what I look like,” she observed. Correctly, of course.</p>
<p>“Yes. I have read articles on astrobiology that say that the portrayal of extra-terrestrial life forms in films is sort of wrong because it is unlikely that aliens would have human like forms. Yet you look so remarkably human!”</p>
<p>“What you read is true, but we have no fixed form. We can imitate any physical object or life-form. We can’t do it on a fine enough level to imitate hair, though, the result of which is there for you to see.”</p>
<p>Before my already bewildered mind could process this completely, I became aware that the continuous sound of writing that had stayed with us throughout was now gone, and the only noises to be heard were coming from the city outside that was slowly waking up. I looked at the man and he seemed to be looking at the picture on his desk. He had a piece of paper on his desk and he seemed at a loss for words.</p>
<p>Presently I heard another kind of sound entirely. Footsteps.</p>
<p>“His seconds must have arrived,” said my guide. Galois must have heard the steps too, for his manner became decidedly more rigid. He glanced at the clock, and let out a small sigh. The steps had now reached what I imagined would be the staircase outside the door. Soon, there was a knock on the door and he got up to answer it. There were a few words in French, and I looked at my guide, nonplussed.</p>
<p>“You don’t know French, do you?” she asked, half-mockingly. “Let me translate. They told him that it is now time to go, and he asked for five more minutes.”</p>
<p>Galois proceeded to arrange all the papers into piles and neatly packaged them. After sealing the packages with a little bit of wax from the candle and his own ring, he addressed each of them separately. There was one package that was much larger than the rest and this he addressed to one Auguste Chevalier. The rest appeared to be individual letters. After putting all the work of the long night aside, he picked up the last piece of paper and quickly wrote a few lines, keeping it very close to himself while he wrote it. His repeated glances at the picture in front of him were enough to tell me that it was she that the letter was meant for. He finished writing it at almost the same time as the next knock on the door. He sealed this last letter too, and then went to answer the door again, returning with a man slightly older than him. He showed this man the results of his night of labour.</p>
<p>I hadn’t noticed when my guide had left the window to stand beside me.</p>
<p>Galois finally spoke.</p>
<p>“I have written addresses on all these packages, please get them delivered, won’t you?”</p>
<p>He was speaking English all of a sudden!</p>
<p>“No, he’s not. I am translating for you in real time,” whispered my guide, strengthening my assumption that she could read minds. “But listen to what they are saying.”</p>
<p>“This letter is the most important of the lot,” Galois continued speaking, handing the last letter to the other man. “Get it to her and ask her to remember me, please?”</p>
<p>“I will give it to her with my own hands, Evariste. There is the one big package, can I ask who will receive that?”</p>
<p>“Monsieur Chevalier. It contains my life’s mathematical work condensed to a few hundred pages. I do hope he is able to understand the work that I have done and ensure that it reaches the proper people.”</p>
<p>“Don’t worry about that, Evariste. I will personally ensure that your work isn’t lost, and I will help M. Chevalier do full justice to it. The world shall know about your contributions to mathematics.”</p>
<p>He didn’t say anything in reply, but the look in his eyes showed that this statement meant a lot to him. He started going around the room, pausing at various points and trying to take in as much of it as possible. His air was that of a man who was seeing his own home for the last time before going away on a long journey.</p>
<p>“I tried to write everything down, as far as possible. I don’t know if I did a good enough job. I hope there’s someone who profits from all the mess,” he said.</p>
<p>“Let’s go. It’s already time,” replied the other man. I thought his voice shook just a little.</p>
<p>“You might think a man who is going to his death could enjoy one last look at his quarters without being asked to hurry up!” replied Galois, simultaneously picking up the gun. There was no humour in the voice.</p>
<p>“Are you sure you want to do this?”</p>
<p>“We have had this conversation before, Emmanuel. It is a question of my honour, and I shall go and face that crooked fool even if it means my death, which it probably will for he has unfortunately always been a much better shot than I could ever be.”</p>
<p>He took one last glance at the room before speaking again.</p>
<p>“There’s nothing of value here… except that picture. Send it to her along with the letter please.”</p>
<p>“As you wish, Evariste.”</p>
<p>With that they left the room, locking it as they went. Perhaps fittingly, the candle went out again just as they left the room.</p>
<p>“Now what?” I asked my guide.</p>
<p>“Do you want to see how it finally ended?”</p>
<p>“I don’t want to, but I am too far in to not see it.”</p>
<p>“Come on then.”</p>
<p>A horse carriage had stopped just below the window. I guessed they were going to the rendezvous on that carriage, and evidently my guide thought so too, for sje jumped onto the roof of the same and I followed suit. After some time on the cobbled streets of 18th century Paris, we arrived at a small clearing just outside the city, where there were already five men waiting. Galois and his friends quickly got out of the carriage and walked towards the other people while we followed closely behind. There were hardly any words exchanged. The man who appeared to be the leader of the other group first shook hands with Galois, and the other people shook hands with each other. Then they marked off two lines about thirty metres from each other and this was double checked by one person from each group. Galois and the other leader took their positions at the two marks. They both had guns in their hands.</p>
<p>The man who appeared the oldest in the small congregation then spoke.</p>
<p>“Are the contestants ready?”</p>
<p>They both nodded their heads.</p>
<p>“Bow to each other, please.”</p>
<p>They bowed.</p>
<p>“Let the duel begin.”</p>
<p>The other man fired first, and missed by a whisker. Galois fired next, but missed by a fair margin. From the look on his face, he knew at that very moment that he had lost his chance.</p>
<p>One more shot was all it took and suddenly Galois was buckled over clutching his abdomen. Blood dripped from the wound, and it wasn’t long before he collapsed. His friends rushed to him and tried to block the wound, but it was pretty clear that all attempts to save him were in vain – the shot was too good and there was no hope of saving him using the medical knowledge of the century I was now in.</p>
<p>It wasn’t long before Evariste Galois departed this world.</p>
<p>While his friends carried the body away, my guide and I remained at the spot where he was shot. None of us spoke for a very long time. I couldn’t read her mind, of course, but I guess we were both wondering about the tantalizing possibilities of what could have been – thoughts that invariably come when someone that brilliant dies that early.</p>
<p>“You know what, I always wanted to visit this day because I thought I could maybe change the outcome by, I don’t know, somehow stop him from reaching the appointed place. Watching while the events happened made me feel so helpless!”</p>
<p>“Yes, it is heartbreaking! To see it happen in front of your eyes and yet being powerless to stop it from happening!”</p>
<p>No one spoke for a few minutes, until she broke the silence again.</p>
<p>“This is enough for today I guess. Let me get you back to your timeline. Next time let’s choose something less bloody.”</p>
<p>The infinite possibilities that this opened up was enough to take mind away somewhat from the brutally sad event that I had just witnessed.</p>
<p>“So I will be seeing you again.”</p>
<p>“Yes, of course. I will get in touch. But now, close your eyes and hold my hand and let’s go back to where we came from. You shouldn’t be here longer in a different timeline.”</p>
<p>I complied, and for the second time in my life her long fingers closed around mine. There was again the impenetrable darkness and feathery lightness that I had felt while starting my extraordinary trip to Paris. Soon, however, I found that I had reached familiar surroundings, the same secluded spot in the grounds of my building from where we had left. I was alone and my hand was clutching only air – my guide was gone. As I absentmindedly made my way back to my room, deep in thought about Galois and the events that I had just witnessed, it occured to me that he had been younger than me when he died. He was barely old enough to be called a man! If it is true that at the point of death our whole life flashes in front of us, it must have been a brief but brilliant flash for the tragic figure of Evariste Galois.</p>Sagnik Bhattacharyasagnikb@umd.eduMathematics, aliens, history, time-travel rolled into one story based on true events.Volumes of spheres in discrete metrics2019-04-20T08:30:00+00:002019-04-20T08:30:00+00:00https://sagnikb.github.io//blogposts/leemetric<p>Recently I co-authored a <a href="/assets/pdfs/Lee_metric.pdf">paper</a> with <a href="#">Prof Adrish Banerjee</a>, which we submitted to <a href="https://2019.ieee-isit.org/">ISIT 2019</a>, and I thought I would go over the key ideas in the paper as a blog post. Our main concern was to find out the volume of spheres in discrete metrics, and applications of the method to the Lee metric.</p>
<h1 id="background">Background</h1>
<p>Why would it be an interesting problem to talk about the Lee metric and bounds of the kind that we were looking for? It turns out that there are quite a few applications for results of this sort.</p>
<p>A bound on the volume of a sphere in a discrete metric is used to find bounds on the limits of reliable communication over a channel (more on that later). We actually see this in the paper itself - the moment we obtain an upper bound on the volume of a sphere in the Lee metric, we could use known results to immediately find analogues of the binary Hamming, Elias-Bassalygo and Gilbert-Varshamov bounds in the case of the Lee metric (more on all of these later). The basic utility of bounds on the limits of reliable communication over a channel should be clear - it gives people who design communication protocols something to aim at, and if such a design hits the upper bound, we know that one cannot do better, at least from the rate point of view (one could still improve things like encoding-decoding efficiency etc.). There are other examples too - consider the result of <a href="/assets/pdfs/Shared_randomness.pdf">another paper</a> that I was a co-author on that was also accepted to ISIT, where we showed that if the capacity under vanishing average error of a channel does not equal the zero-error capacity (both of these terms will also be talked about later in slightly more detail) of the channel, we need at least <script type="math/tex">\log(n)</script> bits of common randomness, where <script type="math/tex">n</script> is the blocklength of the code, to communicate reliably at capacity over the channel. Common randomness being a valuable resource (it is hard to generate and distribute such randomness), such a bound is good to know. How to show such a gap exists? We need an upper bound on the capacity of the kind that our results help us find.</p>
<p>This is the thing that brought our attention to this problem in the first place. We already knew the common randomness result, and we knew that the gap exists for the Hamming metric, but we thought it would be interesting to show that the gap exists for a more general channel - enter the Lee metric.</p>
<h2 id="why-is-a-metric-important">Why is a metric important?</h2>
<p>Before proceeding further, let me quickly talk about metrics and why they are useful in information and coding theory. So a major subfield of information theory involves finding the limits of reliable communication over a particular channel. Now <em>reliable</em> can be defined in various ways - two of which are</p>
<ul>
<li>we want the <em>average</em> error to go down to zero when we use the channel multiple times</li>
<li>we want the error to be <em>exactly</em> zero for all channel uses</li>
</ul>
<p>In most cases, the maximum rate at which we can transmit information in the first case will be higher than the maximum rate for the second case. Now, any transmission is done using an <em>encoder-transmitter</em> pair and a <em>receiver-decoder</em> pair, with a (possibly noisy) channel in between. In recovering the transmitted data from the noisy data at the receiver, it makes intuitive sense that the decoder should choose the most probable transmitted codeword given the model of the channel noise and the received data, an idea called <strong>M</strong>aximum <strong>L</strong>ikelihood (ML) decoding. This is where the metric comes in. If the channel noise model is sufficiently ‘nice’ we can replace the ML decoding at the decoder by using a metric (which is just a measure of distance) such that choosing the nearest neighbour in the set of all possible transmitted codewords to the received data, we get the same result as what we would get if we were to use ML decoding. When this happens, we say that the metric is <em>matched</em> to the channel under consideration. Note that the metric that will be used depends crucially on the encoder, noise model of the channel, and the decoder. This idea will become clearer in the next section.</p>
<h2 id="the-hamming-and-the-lee-metrics">The Hamming and the Lee metrics</h2>
<p>Two metrics that have been studied relatively deeply in the literature are the Hamming metric and the Lee metric. The Hamming metric is a very simple metric - given two <script type="math/tex">n</script>-length words <script type="math/tex">a</script> and <script type="math/tex">b</script>, it counts the number of differences between them. It is useful in cases where, given any two distinct <em>symbols</em> (not words) <script type="math/tex">a</script> and <script type="math/tex">b</script>, the probability that the channel corrupts <script type="math/tex">a</script> to <script type="math/tex">b</script> is the same, and this probability remains the same if the pair <script type="math/tex">(a, b)</script> is replaced by any other distinct pair <script type="math/tex">(a',b')</script>. Even though this metric is really simple, it can be used to describe a large class of encoder, decoder and channel combinations, and thus has been the most well-studied metrics in information and coding theory. The Lee metric is useful whenever symbols closer together are more probable to corruption than symbols further apart, and the decision of the ‘closeness’ of two symbols is made by arranging the symbols in a circle (for example, the first symbol and the last symbol are considered adjacent, which wouldn’t be the case if they were on a line).<sup id="fnref:1"><a href="#fn:1" class="footnote">1</a></sup></p>
<h2 id="rate-distance-trade-offs">Rate-Distance Trade-offs</h2>
<p>Another interesting concept is that of a rate-distance trade-off. It again makes intuitive sense to say that if two codewords are far apart in a suitable metric, then the probability that the channel will confuse the two is low. So, to protect against as many errors as possible, we want our codewords to be as far off from each other as possible. However, in a finite space, we can only pack so many codewords that are all pair-wise more than a given distance away. So, given the number of errors a code can handle, we get a bound on the number of codewords we can use. If we use <script type="math/tex">n</script>-length code, the number of such codewords grows exponentially in <script type="math/tex">n</script>, and therefore it makes sense to define the rate as <script type="math/tex">\log \mathcal{M}(n)/n</script>, where <script type="math/tex">\mathcal{M}(n)</script> is the number of possible <script type="math/tex">n</script>-length messages. In the case of the error being exactly zero, we want there to be no codewords in a sphere around a given codeword. For average error going to zero, we can allow small intersections between the spheres. Note that this means that finding how many <script type="math/tex">n</script>-letter words are there in a neighbourhood of a given radius around a point is going to be important because we have to ensure that none of these is part of the code that we are going to use.</p>
<p>In the rest of this post, we will be concerned with only zero-error information theory, the second error criteria used above. In this case, we know a lot for the Hamming metric - given a particular rate, we know that a particular rate is achievable (that is, we can construct a code that achieves that rate), and we know that some rates are not achievable. The achievability result is via the Gilbert-Varshamov bound, while there are several upper bounds known (like the Hamming, singleton and Elias-Bassalygo bounds). Unfortunately, these do not match, and a gap exists <sup id="fnref:2"><a href="#fn:2" class="footnote">2</a></sup> which has been an open problem for quite some time now.</p>
<figure>
<img src="/assets/images/bounds.svg" width="100%" alt="Binary Hamming metric" /><br />
<div class="divider"></div>
<figcaption> What the bounds look like for the Hamming metric. The Gilbert-Varshamov bound is the achievability result. The rest are upper bounds. Note the gap between them. </figcaption>
<div class="divider"></div>
</figure>
<p>In the case of the Lee metric, far less is known. We saw above why it is important to bound the number of words that are within a given distance away from a point, a quantity that we will call the size of a ball in the metric. The main difficulty in the case of the Lee metric is that it is much harder to find the size of a ball in the Lee metric than it is in the Hamming metric. Assuming we know the size of the ball in the Lee metric, a formula for each of the rate-distance bounds mentioned above appears in a wonderful book - <em>Algebraic Coding Theory</em> by Elwyn Berlekamp.</p>
<p>So now, the issue is to figure out what the size of a ball in the Lee metric is. This is the main content of our paper. Once we had this result, we were able to use Berlekamp’s work to immediately obtain the above-mentioned bounds for the Lee metric.</p>
<h1 id="our-approach">Our approach</h1>
<h2 id="first-steps">First steps</h2>
<p>The first step was essentially a rediscovery of results that were known when Berlekamp wrote his book in the 60s. The idea is that one can think of this problem as estimating a particular coefficient in the expansion of a particular generating polynomial for the metric. To gain some intuition for this, consider the simple case of the binary Hamming metric, and suppose we need to find all the <script type="math/tex">n</script>-letter words that are a Hamming distance <script type="math/tex">d</script> away from the word <script type="math/tex">(0, \ldots, 0)</script><sup id="fnref:3"><a href="#fn:3" class="footnote">3</a></sup>. The Hamming metric just says that if there are <script type="math/tex">d</script> locations where two equal length words differ, the Hamming distance is <script type="math/tex">d</script>. The required number of words is given by the number of choices of <script type="math/tex">d</script> positions out of <script type="math/tex">n</script> that will have <script type="math/tex">1</script>’s, which is simply <script type="math/tex">\binom{n}{d}</script>. This is also the coefficient of <script type="math/tex">x^d</script> in the polynomial <script type="math/tex">(1+x)^d</script>. One simple way to understand the equivalence is that this coefficient also involves the choice of <script type="math/tex">d</script> <script type="math/tex">1</script>’s from <script type="math/tex">n</script> choices.</p>
<p>This idea generalises to the Lee metric. Say the alphabet size in the Lee metric is <script type="math/tex">5</script>. Then the generating polynomial is given by <script type="math/tex">(1 + 2x + 2x^2)</script>, and the problem of finding the size of the Lee ball of radius <script type="math/tex">d</script> reduces to finding the coefficient of <script type="math/tex">x^d</script> in <script type="math/tex">(1 + 2x + 2x^2)^n</script>. Now, this is not an especially easy problem, because no such simple idea as the one in the case of the Hamming metric works in this case. There exist formulae for this (also known from the 60s), but they are a) difficult to use and b) do not give much insight to the problem. I was at this time doing an online course on analytic combinatorics on Coursera, and there they talked about problems of this form and how to find analytic solutions for the same. I learned a lot of interesting new approaches to solving problems like this (in fact the first time I thought about the generating polynomial method of solving the problem was while doing the course), but ultimately in turned out that the techniques in the course weren’t well suited for this particular application (in particular because here we need to fix <script type="math/tex">n</script> to be a particular finite value and there one could take the limit as <script type="math/tex">n \rightarrow \infty</script>, and the forms of the polynomials that we were working with could not be handled very easily via the techniques proposed in the course).</p>
<h2 id="a-question">A question</h2>
<p>While spending a few weeks trying to use techniques from analytic combinatorics and ruling them out one by one, I noticed that even the Hamming metric calculations are done using assuming that we are finding the coefficient of some <script type="math/tex">x^d</script> where <script type="math/tex">d</script> is then the location of the largest coefficient. I figured it would be a start if we could at least figure out where the maximum coefficient would be, we would at least know the parameter range where we would be working. After a few days wrestling with this problem, I seemed to have guessed a solution to it but was not able to prove it myself. So I posted the following question on the Mathematics StackExchange - <a href="https://math.stackexchange.com/questions/3057144/largest-coefficient-in-the-power-of-a-polynomial">Largest coefficient in the power of a polynomial</a>. A couple of answers could be used to get a proof of the maximum coefficient, and I finally hit upon <em>some</em> path of progress - I could use the central limit theorem to find the required coefficients for large enough <script type="math/tex">n</script>. Using Python’s <code class="language-plaintext highlighter-rouge">numpy.polynomial</code>, I was able to get some graphs that showed me how the coefficients were behaving and how the normal approximation looked. I tried to use the results to obtain the bounds that we want, but the results were rather bad - they didn’t even match the known results for the Hamming case, leave alone the Lee metric. When I went to see what the problem was, it was quite apparent - I had been using the log scale to look at the graphs, and the normal approximation was really good close to the peak but was really bad away from it. The log scale hid the fact that a couple of standard deviations away from the peak, the approximation was giving a value many orders of magnitude more than the actual value (say <script type="math/tex">10^{10}</script> times higher).</p>
<h2 id="large-deviations-and-sanovs-theorem">Large Deviations and Sanov’s theorem</h2>
<p>This was a problem that reeks of large deviation theory - but it took me some time to recognise that because I had not used it before. Large deviation theory, as the name says, gives good bounds when one is working more than a few standard deviations away from the central peak.</p>
<p>One of the basic results in large deviation theory is something called Sanov’s theorem. It gives a powerful bound on the size of a set of type classes drawn from a probability distribution. The idea is as follows - suppose we have a discrete probability distribution, like the one that governs what face of a die comes up when it is thrown. Now, if we sample the probability distribution <script type="math/tex">n</script> times (throw the die <script type="math/tex">n</script> times), the outcomes will probably not be distributed exactly like the underlying distribution, but would be something called the empirical distribution - in the case of a die, this would consist of the number of times each number came up when the die was thrown <script type="math/tex">100</script> times. Empirical distributions are also called types, a powerful concept in information theory, with several very nice properties. A type class is the set of all outcomes which have the same type <sup id="fnref:4"><a href="#fn:4" class="footnote">4</a></sup>. Now, given a set of types, Sanov’s theorem allows us to bound the size of that set of types - the number of sequences such that their type is in the set. In our case, the class is just the set of all types such that the expected value with that probability distribution is less than <script type="math/tex">d/n</script>. This could be calculated using the normal approximation, but as mentioned earlier, the bounds obtained from it are pretty poor away from the peak.</p>
<h2 id="convex-optimisation">Convex Optimisation</h2>
<p>Sanov’s theorem requires us to find the type in the set that has the minimum relative entropy with the underlying probability distribution. This is a convex optimisation problem, and one could take its dual. The dual problem was just a complicated single variable optimisation that became something that I could handle without any of the sophisticated methods used to solve convex optimisation problems, using some Python code that ran reasonably quickly. This was part of the problem solved - numerically finding this maximising parameter gave some upper bound on the required coefficient. I wanted something better.</p>
<h2 id="a-weird-function">A weird function</h2>
<p>By the duality properties of convex optimisation it was clear that any positive value of the single parameter that the dual was over would work, so what remained was to choose what the positive value would be as a function of the expected value that was used to define the type class. I plotted out what the optimal value looked like, and it seemed an odd kind of curve. Now the first thing that entered my head was that the negative of the curve looked like a <script type="math/tex">q</script>-th root function, appropriately scaled and shifted. Now, using <code class="language-plaintext highlighter-rouge">curve_fit</code> from <code class="language-plaintext highlighter-rouge">scipy.optimize</code> in Python, I was able to fit it to other functions, like polynomials or exponentials, but either they were very sensitive to parameters or were not very good fits - something that can be quantified using the output of the <code class="language-plaintext highlighter-rouge">curve_fit</code> function. Anyway, it turned out that the best fit among all the functions I tested was indeed given by that weird function, and that’s what found its way into the paper. We also found that perturbing the optimal values for the constants in this fit did not affect the final result much - so this fit was robust in some sense. If you look at the curve and some other functional form comes to your mind, I would love to know it!</p>
<h1 id="final-thoughts">Final thoughts</h1>
<p>Once we obtained the formula for the size of the Lee ball, all that remained to do was to substitute the result in the formulae from Berlekamp’s book and plot the results. The results for <script type="math/tex">q = 6</script> are shown below. We also verified that <script type="math/tex">q = 2</script> with our method gives the same results as the binary Hamming case pointed out above - one way in which we were able to be somewhat sure that the method works.</p>
<figure>
<img src="/assets/images/Leebounds.svg" width="100%" alt="Lee metric for q=6" /><br />
<div class="divider"></div>
<figcaption> What the bounds look like for the Lee metric with alphabet size of \(6\). Note a few similarities with the Hamming case. The GV and EB bounds agree on where the rate is zero (\(\delta = 1.5\)). There is again a gap between the upper and lower bounds, and the EB is again the tightest upper bound.</figcaption>
<div class="divider"></div>
</figure>
<figure>
<img src="/assets/images/comparisonbounds.svg" width="100%" alt="Comparison with the Hamming case" /><br />
<div class="divider"></div>
<figcaption> Our technique (markers) compared with the known results for the Hamming metric case (lines). </figcaption>
<div class="divider"></div>
</figure>
<p>If you are wondering what we used to make the graphs, there’s a very nice way of using Latex in <code class="language-plaintext highlighter-rouge">matplotlib</code> in Python, and <code class="language-plaintext highlighter-rouge">matplotlib</code> can also be made to output the graphs in SVG, for the rather good looking graphs that you see both in this post and the paper.</p>
<p>If you are interested in looking at out code, it is all reasonably well documented and <a href="https://github.com/sagnikb/LeeMetric">uploaded on Github</a>. If you have any thoughts about this work, use the comment section below or get in touch via email. Provided things work out, I might be presenting this at <a href="https://2019.ieee-isit.org/">ISIT 2019</a>, so this might not be the last time that this paper is featured on this blog.</p>
<hr />
<div class="footnotes">
<ol>
<li id="fn:1">
<p>Brief side note here. One might wonder about the metric that describes the situation where ‘closeness’ is defined by arranging on a line and not a circle. In this case, it can be shown that there are no channels that are matched to this metric, and so, even though the metric is mathematically interesting. it doesn’t concern us that much here (it’s not just mathematically interesting, but a situation where it would be of interest to an information/communication theorist would be too big a detour for this blog post.) <a href="#fnref:1" class="reversefootnote">↩</a></p>
</li>
<li id="fn:2">
<p>In the Hamming case the gap exists for alphabet sizes less than <script type="math/tex">49</script>, above which there is a construction based on algebraic geometry that closes the gap. <a href="#fnref:2" class="reversefootnote">↩</a></p>
</li>
<li id="fn:3">
<p>Point to note - for ‘nice’ metrics, one can always translate in this way to calculate the size of the neighbourhood around zero. <a href="#fnref:3" class="reversefootnote">↩</a></p>
</li>
<li id="fn:4">
<p>One of the cool properties of types is that all members of the type class have the same probability. <a href="#fnref:4" class="reversefootnote">↩</a></p>
</li>
</ol>
</div>Sagnik Bhattacharyasagnikb@umd.eduOn the paper 'A method to find the volume of a sphere in the Lee metric, and its applications'One More April Puzzle2019-04-01T06:30:00+00:002019-04-01T06:30:00+00:00https://sagnikb.github.io//blogposts/One-More-April-Puzzle<p>On a <em>melancholy</em> spring evening, you doze off. When you wake up, you find yourself in mathematical utopia - every plane surface is perfectly plane, and ever sphere perfect. In front of you is a perfectly vertical wall that is exactly perpendicular to the infinite plane that you are standing on. On the wall, there is a strange shelf, and lying around are exactly <del>six</del> seven spherical balls of lead. You pick up the one on the ground and ponder about what you’re seeing. The wall is shown in the diagram below.</p>
<p><img src="/assets/images/april.svg" alt="" /></p>
<p>You note the date, and it is April 1, 2019. You realise that if you find what the thing in front of you means, you’ll be able to cause a new <em>renaissance</em> in your own world when you wake up. It’s not long before you are snatched out of this utopia, this Garden of Euler - so think fast!</p>
<p>Oh yes, this world is non-ideal in one way - every collision is completely inelastic.</p>
<h2 id="hint">Hint:</h2>
<p><img src="/assets/images/rhino.png" alt="" /></p>Sagnik Bhattacharyasagnikb@umd.eduOne more special puzzle to celebrate April Fools DayShannon’s Channel Coding Theorem2018-07-27T08:30:00+00:002018-07-27T08:30:00+00:00https://sagnikb.github.io//blogposts/cool-new-way-to-understand-Shannon-result<p>Let me start with some quick praise of MIT and its educational outreach programs, mainly via <a href="https://ocw.mit.edu/index.htm">MIT-OCW</a> and MITx courses on <a href="https://www.edx.org/">edX</a>. These have been an integral part of my college life and as noted <a href="/blogposts/Why-quantum">here</a>, are one of the reasons I am studying the things that I do. <sup id="fnref:1"><a href="#fn:1" class="footnote">1</a></sup></p>
<p>Coming to the point - I was going through a course on Quantum Error Correction, Complexity and Fault Tolerance offered by MIT on edX. Even though I wasn’t able to complete it on time because I discovered it very late, I learned a lot from that course, one of which I will talk about in this post.</p>
<p>So Prof Isaac Chuang wanted to quickly explain the point of Shannon’s Channel Coding theorem in order to draw connections with von Neumann’s pioneering observations in fault tolerant computing, and he came up with an interesting way to put it that I hadn’t explicitly thought about before.</p>
<p>A quick description of the setting of the channel coding problem: suppose Alice and Bob are trying to talk to eachother and they have to do so over a noisy channel (which may simply be the air from Alice’s mouth to Bob’s ear in a noisy marketplace). Somehow, Alice and Bob must ensure that they are able to communicate with low rate of error even over such a channel. Shannon’s channel coding theorem tells us something non-trivial about the rates at which it is possible to communicate and the probability of error involved, but to understand why it is so cool let’s spend some time imagining that we don’t know what Shannon’s result is and think about what we might intuitively expect to happen.</p>
<p>Imagine a professor lecturing in a noisy classroom. We might expect that if he speaks faster and faster, then it will become more and more difficult to understand what he’s saying, that is, the probability of error will increase. He might need to repeat what he said, but that would only reduce the rate. Therefore, it is reasonable to expect that as the rate increases, the communication becomes more and more error-prone. If we plot the probability of error versus the rate of speaking, intuitively we might expect to see something like this…</p>
<figure>
<img src="https://lh3.googleusercontent.com/NhANZjrNhLam1Sz4DaGCtKQVTOsJg5KCB_JI47W0RU_ECyYF99csfK2R0P82f1rGlmZ2MikhUJLYT6U5vjQZf1d_RjmefLnpc3v5fyPwxWMOfitteGD_QrTXO2yqfvx1X6aTEh2Y-NY=w2400" width="70%" alt="Intuition says..." /><br />
</figure>
<p>However, Shannon showed that this is not the case. His result basically showed that there is a rate C, called the capacity, associated with a channel which precisely characterises what rates are possible and what are not. Every rate below C can be achieved with negligible probability of error (in the limit of large block-lengths, a subtlety that, along with several others, I will ignore to make in order to illustrate the point here) and every rate higher than C will result in error probability bounded away from zero (that is, always greater than some number which is greater than zero). In fact, it was shown later that when the rate is higher than C, the probability is actually very close to one, and if we re-draw the graph, we get something like this…</p>
<figure>
<img src="https://lh3.googleusercontent.com/2RJYbUwSOYv2_yUYnaNxmyQt6ZiFYQMoLn9bY1DzFPzKkwLN3WvN6PxeXb2ON8WoaS-4vHTw0RaFVmP_x6Yko55HCpIsD8f9yvljW1676NphcTI_MtBfEejRO2-iXgDXEUdRdpaJLgM=w2400" width="70%" alt="Intuition says..." /><br />
</figure>
<p>This is precisely the non-intuitive content of Shannon’s channel coding theorem. A similar result was derived by von Neumann where he showed that as long as the basic gates used in constructing a computer are more reliable than a certain threshold, one could make a highly precise computer. What prompted him to think in this direction and why that result is hugely important today is an interesting tale in its own right, and one that I won’t be attempting here.</p>
<p>The impact of Shannon’s results can hardly be overstated - one could argue that his highly original ideas ultimately kickstarted the communication revolution that has ended up giving us mobile phones, the Internet etc. This way of looking at it gives us a way to appreciate Shannon’s result without going into any math whatsoever.</p>
<div class="footnotes">
<ol>
<li id="fn:1">
<p>Quick side note: that link is a bit backdated at this point. Information and coding theory have gained a good amount of real estate since then as far as my interests are concerned, but that is another story for another day. <a href="#fnref:1" class="reversefootnote">↩</a></p>
</li>
</ol>
</div>Sagnik Bhattacharyasagnikb@umd.eduThe (highly) non-trivial content of Shannon's Channel Coding theoremTravelogue – Old Bangkok2018-06-25T08:30:00+00:002018-06-25T08:30:00+00:00https://sagnikb.github.io//blogposts/bangkok<p>The whiff of Sanskrit and by extension my homeland of India that I got as our flight touched down at the Suvarnabhumi International Airport of Bangkok would only get stronger as we saw the sights in the city. If the decorations in the airport seemed vaguely similar to certain temple decorations in India, the paintings depicting scenes from the Ramayana on some of the walls of the temples of old Bangkok almost made me believe I was in India. Such is Bangkok and Thailand – a country with a culture that is at once very similar to and very different from the familiar surroundings of India.</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/GIUX5iYPB9_n5c_u8lAgxdY-ofNle5fPanHcyrCkNN35bwSi84EA8jwjsJWR1C2vf7SgcHqG_wuGe1yuWtMbqi898RHzMCrhSC17tsbKsH6t5dvHJX9HwQEVSP5Z46lkkoTdKtxiL1s=w2400" alt="A temple complex" />
</figure>
<p>On second thoughts, I should not have been surprised to know that King Rama still reigns in Thailand and that the old capital of Thailand is called Ayutthaya after Ayodhya, the birthplace of Lord Rama in Hindu mythology – after all, the largest Hindu temple in the world is not in India, but is the Angkor Wat temple in modern day Cambodia. It is a pity that the history of the golden age of the south Indian empires like the Cholas is not taught much in Indian history at the school level, for it is partly because of them and their empires that stretched across southeast Asia that many countries in this part of the world still contain strong imprints of medieval Indian culture. Thailand is one of these countries – even though the dominant religion now is Buddhism, there are fingerprints of Hinduism all over the country. Apart from the Sanskrit place names and the wall paintings, this is also somewhat visible in the art styles and traditional clothes of the country. Analysis of the art forms and other historical evidence suggests that even Theravada Buddhism was introduced to Thailand via the Gupta empire.</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/r-L6poumybRYs_LwlvPhIb-fUGLHNesCXOC2n8kAaZOR3_biaffKmjGooylTFY_RX_iBvQ7j8NTCtQLu2GCMf57zQ6ryN7FLhrtXNQapeQV6E7AffU05zqU_f1KeYgxixzvrKfDmgnM=w2400" alt="Wat Pho Reclining Buddha" />
<figcaption>
The Reclining Buddha of the Wat Pho Temple Complex, Bangkok. The people in the frame can give you an idea of how big the statue is. ‘Architecture as power’ was nowhere more apparent than in front of the gigantic reclining Buddha in the Wat Pho temple complex.
</figcaption></figure>
<p>The old part of the city of Bangkok is where these similarities could be seen the most. Apart from the paintings from the Ramayana on the temple walls as mentioned earlier, it was interesting to note that the Buddha statues and the style that they were painted in. Unlike the Chinese style, these idols had faces that looked familiar from several places in India, like the Ajanta and Ellora caves. Of course, the ornamentation and the use of gold was something that is not seen in the Ajanta and Ellora caves, and neither is the elaborate decoration seen in some of the Buddha statues.</p>
<figure class="half ">
<img src="https://lh3.googleusercontent.com/lnbeOSvCYyd3gU6OHRk0M-ROM84Xtllkf2oXzMt0-pY-Iv4_8cYb3cjwmmwpw9IK9Ztx67SpJHcNGQzKIvmKhJK3vu7doJ3GBshIpP_GnPuIMq922f3qjGWYV-crkrN83i6G88kpNIs=w2400" alt="One of the more Indian looking Buddha idols" />
<img src="https://lh3.googleusercontent.com/NdpeVjDoHXj7YRdKEvsiIq5ezO4KNRc1H0nSvYHVyoBT4N8Ti6dpfXw3kd46Z189IweRM54T4ERv5oSOEBiyxy-l2A8eL1N1h9oWOR68VdEnghoDpYi8T30BRMcxCMDDbAh7f_Ap63s=w2400" alt="A series of Buddha idols in one of the temple complexes" />
<figcaption>One of the more Indian looking Buddha idols along with a series of Buddha idols in one of the temple complexes
</figcaption>
</figure>
<p>The Grand Palace itself has a curious mixture of Asian and European styles that I hadn’t before – in particular, the facade is in European style, while the roof decorations are from South-East Asia. They reminded me of the design of the Khmer civilization buildings in the Age of Empires II Asian Dynasties expansion.</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/r9xmxY-v8-jl5wlm36bpgmaGyhaUUbrmryOXX5dx1PXXqxmjzUHZcxsmot9yNklRNNtLTeItCsUppH_VGrBPFJ-CK7DLGXMRtvHAhhM9B-QB0nSjqJ2LALinmO56kZS8DwH2ivt6Odc=w2400" alt="The Grand Palace, Bangkok" />
<figcaption>
The Grand Palace, Bangkok
</figcaption></figure>
<p>Much of my knowledge of the history of Thailand comes due to a particular tourist attraction that is strongly recommended that you visit if you are going to Bangkok – Jim Thompson’s house. Jim Thompson was an American businessman and architect, who built his house in central Bangkok by putting together six traditional Thai houses and interconnecting them. It is now a museum, housing the substantial collection of art that Mr. Thompson accumulated before his disappearance in 1967. One can find paintings and sculptures from all over southeast Asia in that house, and even if you’re not interested in art or history, you can always check out the tropical garden within the compound. It is here that I found some official source to confirm that Theravada Buddhism was introduced to Thailand via the Gupta Empire of India, which goes a long way to explain why their Buddha statues have faces resembling Buddha faces in the Indian style.</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/3Esh3e5zkd_gGbfMewkdNg4vMLVg_e-VrVosqNKa7xa2cN-zzDzR7lgUYzVQPt8IL4Di3MbrLWSQ2oNyISxIySHUBw4ylCMMtXja8AOPZCSMjDtteLLdzLe0XU1Cxb5zM-1Lg3s-mK0=w2400" alt="A small part of Jim Thompson’s substantial art collection" />
<figcaption>
A small part of Jim Thompson’s substantial art collection
</figcaption></figure>
<p>Something else that I tried during our stay in Bangkok was some interesting food. The most interesting was a serving of traditional pineapple rice, served in an actual pineapple, and I am pleased to say that it tasted delicious! I also had a santol fruit juice – one of the first times that I needed to Google a fruit’s name before actually having it.</p>
<figure>
<img src="https://lh3.googleusercontent.com/U-oUwNz566HOp28TJkywRbWLIHrYrS_tnmGaWYzXhMeNA4RolqRuFA2KZCd_W0AjRlwqiGzbBbZvbMe7vwsoLHeGwLwlQY7BjV-xMNsiU7FMefPzuLGbm6zxVFgPJEtlVKkxo6u7aEg=w2400" alt="santol fruit juice" />
<figcaption>Santol fruit juice</figcaption>
</figure>
<figure class="">
<img src="https://lh3.googleusercontent.com/3vndJwsgRV1ws_SJlLJi9b6vseSltDxnLXdMubME1BwH5Foukr5jKL9152DtUm6nCUda1UeXYLyB3ZlnDx8qIZKvEaqdqiAC8I-90XnMMTeMyZK3hzxMrXoXOGtP3SMVmPco1p30bEY=w2400" alt="pineapple rice" />
<figcaption>
Pineapple on pizza may be an abomination, but pineapple rice, Thailand style, is definitely not!
</figcaption></figure>
<p>The coolest memento I have from my time in Bangkok is a croaking wooden frog – the croaks really are realistic!</p>
<figure class="">
<img src="https://lh3.googleusercontent.com/NxSBBI_DA2Pv1ZhLRwhJSYKY-34clJJy_r0Nl9yUvc2HBA6JFvrWzSKPNg6tdJ628FlIzNWl-qyG4RQVlSh3ZxwmQ0KWOtaPEEkkGTZnDNGTDJkOr85SbL5aeydHTOVSNgTZH2gYDTA=w2400" alt="wooden frog" />
<figcaption>
I now own this funny little wooden frog that can make pretty realistic croaking sounds. It seems these things are pretty popular in Thailand!
</figcaption></figure>
<p>I had this idea before going to Bangkok that the experience of the city was limited to glitzy shopping malls, spas, and parties, but even though we visited a couple of those places too, easily the most surprising part of the trip was the day, or so we spent exploring the old city. This part of the ‘Bangkok experience’ isn’t emphasized much in the stories that people tell of the city, but it’s certainly worth checking out.</p>
<hr />Sagnik Bhattacharyasagnikb@umd.eduAdventures in the old city of BangkokBinary Search, the Ulam game and Coding Theory2018-05-07T08:30:00+00:002018-05-07T08:30:00+00:00https://sagnikb.github.io//blogposts/Ulam<h2 id="introduction">Introduction</h2>
<p>Binary search allows us to search for an entry in a sorted list in <script type="math/tex">\log(n)</script> time. In this post, I shall talk about a more general version, where the answers to some of the queries used during the binary search may be false (which is the Ulam game) and how that is related to adaptive block codes in coding theory and how these ideas help us.</p>
<p>Much of this is based on <a href="http://www.desjardins.org/david/thesis/thesis.pdf">David desJardin’s PhD thesis</a> which gives a general solution to Ulam’s game. I am writing about it because I had heard of the Ulam game where only one lie is allowed from a friend and was surprised to see the same crop up when learning about coding theory. It is not a very difficult topic while also being an interesting extension of something well known, so I thought I could give an overview of the same in a post such as this.</p>
<h3 id="the-ulam-game">The Ulam Game</h3>
<p>The game 20 questions is a specific formulation of the problem of searching for a marked element in an ordered list of elements and is easily solved by binary search. The game itself is the following:</p>
<blockquote>
<p>Alice thinks of a number between one and one million. Bob can ask up to 20 yes/no questions to Alice, each of which Alice answers truthfully. Bob wins if he can determine which number Alice had thought of by asking just the 20 questions.</p>
</blockquote>
<p>Note that <script type="math/tex">2^{20}</script> is <script type="math/tex">1048576</script>, which is just more than a million. Bob always wins because he can do a simple binary search for the number which will give him the answer in <script type="math/tex">\log_2(2^{20}) = 20</script> steps. The questions, in this case, could be of the form:</p>
<ul>
<li>is the number greater than 500,000? If the answer is no,</li>
<li>is the number greater than 250,000?</li>
</ul>
<p>and so on, halving the range every time.</p>
<p>We can formalize this process of questioning in the following way - we can think of each question as a subset of the positive integers below one million, and Alice’s answer is yes if her chosen number belongs to the set and 0 otherwise.</p>
<p>It is interesting to note that Alice need not decide on her number at the beginning - if all her answers are merely consistent with some number that she chooses at the end, Bob still wins.</p>
<p>Stanislaw Ulam proposed a more interesting version of the above in his autobiography, called the Ulam game.</p>
<blockquote>
<p>Someone thinks of a number between one and one million (which is just less than <script type="math/tex">2^{20}</script>). Another person is allowed to ask up to twenty questions, to each of which the first person is supposed to answer only yes or no. Obviously, the number can be guessed by asking first, “Is the number in the first half million?” then again reduce the reservoir of numbers in the next question by one-half, and so on. Now suppose one were allowed to lie once or twice, then how many questions would one need to get the right answer?</p>
</blockquote>
<p>One clearly needs more than <script type="math/tex">n</script> questions for guessing one of <script type="math/tex">2^n</script> objects because one does not know when the lie was told. This problem is not solved in general.</p>
<p>When I first came across this problem via a friend, the question was to find some lower bound on the number of questions needed. One possible lower bound is</p>
<script type="math/tex; mode=display">\log(n) + \log(\log(n))</script>
<p>where all logarithms are to the base-2. The reason is the following: let us perform the binary search again. In the worst case, we will need <script type="math/tex">\log(n)</script> questions before we get a contradiction. Then we need to find the questions to which the answers were wrong, and since there are <script type="math/tex">\log(n)</script> total questions, we need at least <script type="math/tex">\log(\log(n))</script> steps to find such questions.</p>
<p>It turns out that the Ulam game is equivalent to the problem of showing the existence of adaptive block codes of certain parameters, and this will occupy us for the rest of this post.</p>
<h3 id="adaptive-block-codes">Adaptive Block Codes</h3>
<p>Coding theory deals with reliable communication across a noisy channel. Alice and Bob are communicating over a channel that corrupts some of what Alice sends, and the idea is that Alice encodes the data in such a way that Bob can correctly decode it given some reasonable assumptions about the level of noise in the channel.</p>
<p>We can possible classification of channels is into two categories - with feedback and without feedback. <strong>Coding without feedback</strong> depends only on the input, and the transmitter is assumed to have no information about what is happening at the receiver. In the <strong>feedback</strong> case, the encoder is also considered to have access to the information about whether the previously sent information was received in error or not, and can use this information while sending the rest of the data. In our case, this information is assumed to reach the encoder immediately and noiselessly.</p>
<p><strong>Block coding</strong> is a coding technique where the information to be sent is broken up into blocks. Each of these blocks is mapped to chunks of appropriately chosen transmitted bits to reduce the probability of error. This is in contrast to <strong>convolutional</strong> codes where the data to be sent is encoded into the transmitted bits continuously (without breaking into chunks).</p>
<p>Claude Shannon in his landmark 1948 papers showed that associated with every channel, there is a certain rate of communication called the capacity which characterizes it. He demonstrated that codes exist that guarantee nearly error-free communication at rates below channel capacity. He also showed in 1956 that feedback does not increase the theoretical capacity of the forward channel. However, the use of feedback in coding can give us better encoding and decoding strategies; therefore the question of the existence of adaptive block codes is an interesting question.</p>
<h2 id="the-equivalence-between-the-two-problems">The equivalence between the two problems</h2>
<p>The proof of this equivalence is in the PhD thesis mentioned above. Instead of redoing the proof, let’s see why the equivalence intuitively makes sense.</p>
<p>Let us consider what we are actually doing when we say we want an adaptive block code to exist. The message to be sent has been mapped to a chunk of bits to be transmitted, and these bits are transmitted one-by-one. Once each bit is sent the transmitter receives a signal from the receiver informing whether the bit was in error or not. What we want is that after all the bits have been sent, the receiver must be able to figure out which message was sent. The relation with the Ulam game is now clear - think of the bits being sent as being the yes/no questions in the Ulam game and the response from the receiver being the received answer. If in Ulam’s game a number can be pinpointed using <script type="math/tex">N</script> questions, we can say that a message <script type="math/tex">M</script> among <script type="math/tex">2^N</script> possibilities can also be obtained with certainty after the whole process is over. In both cases, the number of errors/lies is assumed to be less than some number <script type="math/tex">E</script>.</p>
<p>The formal statement of the above is the following.</p>
<blockquote>
<p>There exists a winning strategy for Ulam’s game with parameters <script type="math/tex">N</script>, <script type="math/tex">E</script>, <script type="math/tex">M</script> if and only if there exists an adaptive block code with the same parameters.</p>
</blockquote>
<p>Here <script type="math/tex">N</script> is the number of questions in Ulam’s game and the number of transmitted bits in the block code, <script type="math/tex">E</script> is the number of lies in Ulam’s game and the number of errors during transmission and <script type="math/tex">M</script> <script type="math/tex">= 2^N</script> is the number of possible numbers/messages.</p>
<h2 id="solving-the-ulam-problem">Solving the Ulam Problem</h2>
<p>Next, let’s quickly take a look at how to tackle the Ulam game.</p>
<p>First, state space is constructed using tuples of non-negative integers, and a game is defined on it. The game starts with an <script type="math/tex">(E+1)</script>-tuple of non-negative integers, called <script type="math/tex">s_0</script>. There is a translation operator <script type="math/tex">T</script> defined on the tuples that drops the first number in the tuple and retains the rest, taking an <script type="math/tex">(E+1)</script>-tuple to an <script type="math/tex">E</script>-tuple. Bob partitions <script type="math/tex">s_0</script> into <script type="math/tex">a_0</script> and <script type="math/tex">b_0</script> such that</p>
<script type="math/tex; mode=display">a_0, b_0 \in \mathbb{N}^{E+1}</script>
<p>and</p>
<script type="math/tex; mode=display">s_0 = a_0 + b_0</script>
<p>Alice can choose either one of <script type="math/tex">(T \cdot a_0 + b_0)</script> or <script type="math/tex">(a_0 + T \cdot b_0)</script>, her choice becomes <script type="math/tex">s_1</script> and the process is repeated. Bob wins if and only if sum of elements of <script type="math/tex">s_N</script> is <script type="math/tex">0</script> or <script type="math/tex">1</script>, and a state for which Bob wins is called a winning <script type="math/tex">N</script>-state. If <script type="math/tex">s_0</script> is the <script type="math/tex">(E+1)</script>-tuple</p>
<script type="math/tex; mode=display">(0,0,...,0,M)</script>
<p>then this game is shown to be equivalent to Ulam’s game with parameters <script type="math/tex">N</script>, <script type="math/tex">E</script>, and <script type="math/tex">M</script> defined above.</p>
<p>Using this state space representation, certain states are shown to be winning <script type="math/tex">N</script>-states (for suitable <script type="math/tex">N</script>) in the state space, which are called:</p>
<ul>
<li><strong>singlets:</strong> those tuples for which sum of all entries is 1.</li>
<li><strong>doublets:</strong> those tuples for which sum of all entries is 2.</li>
<li><strong>zero-error states:</strong> these are states where <script type="math/tex">E=0</script> (that is, there are no lies). Then the tuple <script type="math/tex">s=(s_0)</script> and such a state is winning <script type="math/tex">N</script>-state iff <script type="math/tex">s_0 \leq 2^N</script>.</li>
<li><strong>Fibonacci states:</strong> these are explicitly constructed states that are part of a winning sequence of states. A sequence of states is called a winning <script type="math/tex">N</script>-state sequence if <script type="math/tex">s_n</script> is winning <script type="math/tex">(N+3n)</script>-state for all <script type="math/tex">n</script>.</li>
</ul>
<p>The volume of an <script type="math/tex">N</script>-state, defined as</p>
<script type="math/tex; mode=display">\text{Vol}_N(s) = \sum_{J = 0}^{E}s_J\sum_{J'=0}^{J} \binom{N}{J'}</script>
<p>is shown to be at most <script type="math/tex">2^N</script> for winning <script type="math/tex">N</script>-states. This is called the <strong>volume bound</strong>.</p>
<p>Winning <script type="math/tex">N</script>-states that meet the volume bound are either singlets or can be shown not to be winning <script type="math/tex">(N-1)</script>-states and are therefore called borderline winning <script type="math/tex">N</script>-states.</p>
<p>Another result called the <strong>translation bound</strong> shows that if <script type="math/tex">s</script> is winning <script type="math/tex">N</script>-state, <script type="math/tex">N \geq 3</script> and sum of all entries in <script type="math/tex">s</script> is at least <script type="math/tex">3</script>, then <script type="math/tex">T \cdot s</script> (the translation operator <script type="math/tex">T</script> acting on <script type="math/tex">s</script>) is a winning <script type="math/tex">(N-3)</script>-state.</p>
<p>A final result called the <strong>summation rule for states</strong> gives a condition on states <script type="math/tex">s</script> that can be shown to be winning <script type="math/tex">N</script>-states based only on the knowledge that certain states <script type="math/tex">c</script> and <script type="math/tex">d</script> are winning <script type="math/tex">N</script>-states.</p>
<p>We can prove certain states are winning <script type="math/tex">N</script>-states using the above results, in the following way. We start with certain states from the four categories that are known to be <script type="math/tex">N</script>-winning, and put them together using the summation rule to find a <script type="math/tex">(N+1)</script>-winning state. This lets us form a binary tree, with the parent being an <script type="math/tex">N</script>-winning state and the children being <script type="math/tex">(N-1)</script> winning states.</p>
<p>All these results allow us to find winning states. There are also some results that allow us to do the opposite, that is, show that some state is a losing state. This is easy if the state violates the volume bound or the translation bound, and in other cases, we can show results of the following form: if a tree rooted at a particular state is of depth <script type="math/tex">x</script> then the volume of the state is less than <script type="math/tex">v</script>. Thus, in some cases, looking at a state we can predict how deep the subtree at that state is.</p>
<p>Of course, because all three of the starting problems are equivalent, each of the winning <script type="math/tex">N</script>-states we find also gives us a solution to the Ulam problem and shows the existence of an adaptive block code with the same parameters. Killing three birds with one stone.</p>
<p>The rest of the thesis uses all of these results to computationally find optimal codes given the parameters <script type="math/tex">N</script>, <script type="math/tex">E</script>, <script type="math/tex">M</script>. Optimal in this case means that an adaptive block code ceases to exist if the value of <script type="math/tex">M</script> is increased by one. The thesis includes parameters for all optimal block codes up to <script type="math/tex">M = 2^{20}</script>, please check that out if you are interested.</p>
<p>As mentioned above, this whole post is based on <a href="http://www.desjardins.org/david/thesis/thesis.pdf">David desJardin’s PhD thesis</a>. Please look at it for further details.</p>Sagnik Bhattacharyasagnikb@umd.eduBinary search with lies allowed and adaptive block codesApril Puzzles2018-05-05T06:30:00+00:002018-05-05T06:30:00+00:00https://sagnikb.github.io//blogposts/April-Puzzles<p>I made these puzzles for Science Coffeehouse and were released on April Fools Day. Needless to say there will be some relation between the day and the puzzles. Don’t look at the solutions before trying though, they are not very difficult! I received some input for puzzle number three from <a href="http://prannay.com/">Prannay Khosla</a>.</p>
<h2 id="puzzle-1">Puzzle 1</h2>
<p><img src="/assets/images/welcome.png" alt="" /></p>
<h2 id="puzzle-2">Puzzle 2</h2>
<p>You have an important homework to hand over tomorrow, but because you are stuck here at SCH, you can’t write it out. So you explain the ideas to your friend and ask him to type it out and you come here to hang out with the cool guys. After a week, when in true IITK style you will have completely forgotten what the homework is all about, you enbe handed it back with a note to see the instructor. There will be gibberish on the paper:</p>
<p><img src="/assets/images/text.png" alt="" /></p>
<p>You will recall your friend saying that he is a lousy typist, but you didn’t think he could be so bad! You will quickly figure out that he pressed one key among the intended key and its immediate neighbours. The spaces are correct, for he didn’t miss that gigantic key, and so are the numbers, for he loves math. You will need to quickly decipher what you wanted to say, for your marks will depend on you telling your professor what you wanted to say.</p>
<p><img src="/assets/images/keyboard.png" alt="" /></p>
<h2 id="puzzle-3">Puzzle 3</h2>
<p>You are chilling on a long weekend, and because you are hungry you
decide to make a sandwich. You reach for your jar of mayonnaise and lo
and behold a resident of the exotic planet of Xnorlax who you had never
before had the privilege of coming across. Before you can introduce
yourself, the Xnorlaxian challenges to a duel of wits. Needless to say,
you (being very proud of your wits) agree.</p>
<p><img src="/assets/images/xnor.png" alt="" /></p>
<p>The Xnorlaxian claims that xe (the only pronoun Xnorlaxians accept)
has a better calculator than yours. You disagree, and you immediately
want to see what the calculator is. Xe agrees, and xir calculator is
shown below. There are two keys you don’t recognise and xe immediately
notices. Xe says xe’s going to show you the power of the calculator. You
can see all that xe does and the results.</p>
<script type="math/tex; mode=display">% <![CDATA[
21 + 34 + 29 << 2 + 25 >> 1 + 3 << 1 + 4 - 459 >> 3 = 8 %]]></script>
<script type="math/tex; mode=display">% <![CDATA[
13 << 1 + 45 << 1 - 50 >> 2 + 3 >> 1 << 2 + 30 >> 1 = 23 %]]></script>
<p>You suddenly claim that you can do the same using your calculator. Xe
gives you the following calculation - can you quickly use your calculator
to figure out what the calculation below should give?</p>
<script type="math/tex; mode=display">% <![CDATA[
22 + 33 << 1 + 50 >> 1 + 130 << 1 + 9 = ???? %]]></script>
<p><img src="/assets/images/calc.png" alt="" /></p>
<h2 id="puzzle-4">Puzzle 4</h2>
<p>You are an assistant to the brilliant spy Eleanor Shadymiss and you are
on the field trying to track the movements of the great aviator
Eadwaerd Shackleplane who you suspect may be up to some illegal
business. You come across an interesting piece of paper among his
navigation charts and now you and Elli (as Ms Shadymiss likes to call
herself) must glean some information from it. You can clearly see that
some squares are important, but you need to figure out which one is the
secret location!</p>
<p><img src="/assets/images/map.png" alt="" /></p>
<h2 id="puzzle-5">Puzzle 5</h2>
<p>In the beginning there was the Word. This Word gave rise to everything around us. Series of great people have roamed the surface of the earth, inspired by the Word. Of all of them, we remember very few. Puzzles the mind, does it not? You may be the one humanity will remember. Have you the spark that will ensure that we do? Been to places, have you? April be this month. Fooled be he who will not believe it! You need to be the very first to be remembered though, like the great Neil Armstrong, or Newton, or Romulus or Hannibal. Have the courage to act. A mind of your own. Date after date will remember you for that. Time will never forget you. And you will be famous! Location after location will have monuments built in your remembrance. Though the path is complicated, persevere! Be brave, be strong, and come to the place you have discovered! There will we be, waiting to discover you!</p>
<h1 id="solutions">Solutions!!!</h1>
<p>To be updated soon…</p>Sagnik Bhattacharyasagnikb@umd.eduScience Coffeehouse - special puzzles to celebrate April Fools Day