Skip to main content

A tour of the Indian Institute of Science, Bangalore


Last week I had an opportunity to visit the Indian Institute of Science in Bangalore for 5 days, and I must say after the trip, I absolutely regret flunking KVPY! For those of you who don't know, KVPY is an add-on test given by students during their amazing days of JEE Preparation. The point is, I really loved the campus, and the people I networked with. I was present there to attend a Workshop of High Performance Computing, and other related fields to it.


A big thank you to Swapnil Parekh who recommended and invited me to this course. At first, I was really hesitant to participate, since there was nothing in that course that could directly be of use to me, as a student, or as a developer with my current Tech Stack experience. Still, you never know what good might come off it and after all, it's something new that I will learn. It could be a great experience. Yeah I know the reasoning's not pretty convincing, but I went for it anyway.




My review of Bangalore wouldn't really be negative. It's a fairly polite city that could use some more Hindi speaking people. I spent really less time exploring Bangalore as compared to exploring the campus of IISc. I must say, it's a fairly large campus, a little bigger than my home campus, NIT Hamirpur. I had a lot of fun cycling through its many connected lanes!


Let's talk business

Obviously, most of my time was spent on the purpose of the visit, learning about High Performance Computing! I am planning on writing seperate blogs for each of them in detail, so here's an overview.

Parallel Architectures

The course began with a great lecture on Parallel Architecture and the need for it. Since the workshop was open for people from all industries, the introduction was mainly a general topic. I could understand most of it because of my recent Computer Organization course in the previous semester. We were taught mainly about Instruction Level Parallelism, Superscalar Parallelism, Memory Management, etc.
I think these topics would mainly interest my peers from IIIT and NIT since these are all the topics still fresh in our minds. I saw the practical use case of Flynn's Classification Schema in Multiprocessor Architecture. Well, multiprocessors do seem really fun and exciting until you realize that all the parallel processing would go for a toss if you have a lot of inter-related computations on the data. In the hands-on session, there were certain case questions when I thought that it could be better if we could just spare the trouble and simply do it on one processor!
Anyway, AI/ML engineers would know the advantage of so many processors and GPUs. With a lot of data, come a lot of GPU requirements. Not a big word, GPUs are basically used to outsource computations from the main CPU. They are processing units, which are used when we have a lot of data to process.
If you have m different processors and a loop that is running n times, you could parallelize it and from a complexity of O(n), you now have a complexity of O(n/m). More the number of processors, lesser will be the time for computations.

Now the question is, how do we get away by simply using the phrase parallelizing the code. How do you do that?

Parallelization Principles

Once we had studied the architecture, it was time to learn how to structure our algorithm and code in such a way so that we could maximise processor utilisation. The upcoming days also provided a hands-on session to us where we could run our codes on the clusters of the SERC Department of IISc. In certain cases, where our computation is always using a predictable section of data, it is indeed satisfying to see all of the n computations getting executed seperately in one go. But, consider a complex situation such as sorting, and things start to take an interesting turn. 

OpenMP

This was the lecture where I learnt most about processors and cores. What if we could make use of multiple threads on just one processor, using multiple cores of our PCs? Most of the applications that we use do make the use of Quad Core or Octa Core Processors, but our naive coded programs usually run on just one core, on a single thread unless we ask the compiler to graciously do it using a library called OpenMP. Processor will have a shared memory (unless otherwise specified) among its many cores and multiple threads updating the same piece of data will give rise to a race condition. So we will need to define a critical section. Yes, a topic borrowed from my Operating System's course. This course was like a flashback test of my previous semesters on many levels.
I think I had the best time during the hands-on session using this library. 

MPI

Now comes the real deal. A processor, or a cluster's node can have limited number of cores. But if we are talking about supercomputers, (and also clusters), they have multiple processors. In architecture, groups of processors or cores form a node, and each node has an associated memory with it. If we want to use more processors, they will have a seperate memory. So if a data is updated by one processor, the other processor which might need that updated data cannot access it. Unless, we send that data from one processor to another processor using a network which connects them.
That, is why we need the Message Passing Interface (MPI). To specify the data that we need to send and the data that we want to receive. It really has a lot of interesting applications, and people who are coding code to be run on multiple processors, that's the thing for you.

OpenACC

We can also use GPUs for accelerating our computations. OpenACC is a library used just for that. With multiple cores using the same memory, it is kind of like OpenMP, except OpenACC directs the compiler to outsource the computations to the GPU attached. I think this topic would be very important for AI/ML enthusiasts, while training models, or while writing internal code to train models.

Introduction to Big Data

Some brief introduction to Big Data and ML topics and how we can use all the above to get better results spurred my inner curiosity towards this amazing field. Anyway, if you're reading this and you have studied Machine Learning, there's not really anything here that you won't know about. And if you're new to the field, there's still I can't tell you much about it :)


Well, that concluded the Workshop, and we finally left the beautiful city of Bangalore with a ton of knowledge and experience with us.

I would like to thank the Supercomputing Education and Research Centre(SERC) for conducting this workshop, and would also like to thank the professors and industry personnel who contributed their time to this. A special thanks to Professor AdityaProfessor Akhila, Professor Yogesh Simmhan and Professor Govindarajan for this amazing workshop. 
We had a great time!

 




Comments

Popular posts from this blog

i3wm essentials - II

Welcome back! Let's continue this guide with other setup essentials for i3. Enabling Mousetap Chances are that if you're using a laptop, then tapping on the mousepad does not equal a click for you. You need to enable tapping in your config. Fortunately, there is one documentation available that works for majority of the setups. I don't need to explain this one in detail. Here you go: Enable tap to click in i3 . Volume Control This one is simple again. Do you remember the i3 config file I talked about in the previous blog ? All you need to do is go to that file and find the line: bindsym XF86AudioRaiseVolume Just below that line you will find lines with XF86AudioLowerVolume and XF86AudioMute too. Anyway, the truth is, there are 2 sets of lines with these keywords. Chances are that the line: bindsym XF86AudioRaiseVolume exec --no-startup-id pactl -- set-sink-volume 0 +5% Will be uncommented and the line: bindsym XF86AudioRaiseVolume exec --no-startup-id pactl -- set-sink vo

Namaste JavaScript Quick Notes

Note:  Akshay Saini's Namaste JavaScript is probably the best course for JavaScript developers out there. These are my personal notes that I made while watching the course; they serve more of as an online quick reference for my understanding and revision, and I hope it benefits anyone reading it too! Everything in JS happens inside an Execution Context. Before a JS code is run, memory is allocated and variables are set as undefined   , and functions are set as their exact code in the scope within the Execution Context. The global execution context hosts all the global variables and function definitions. An Execution Context has 2 components: Memory, that stores variables and functions; and Code, that reads and executes the code. Call Stack maintains the order of execution contexts. Since JS is single threaded and asynchronous, at one point of time, only one function is executed which is at the top of the call stack. For each function, an execution context is created before executi

i3wm essentials - I (Brightness)

So you have started using i3 and somehow managed to open your browser and almost resumed your normal work.  But wait, the brightness is too much isn't it? Or is it too low? The mousepad used to work fine, but now all of a sudden tapping does not equal click?!  Don't worry.  This blog series will tell you all about the essential setup commands and common shortcuts that I use to navigate my work in i3, and how you can too. Changing the brightness So you just started i3 and you just can't take this brightness setting. You go for your function keys, and damn! They aren't working. Quick fix: Run the following command if you need to change the brightness ASAP. xrandr -q | grep ' connected' | head -n 1 | cut -d ' ' -f1 This will give an ouput that's the name of your monitor.  Use that monitor name here and change the values of brightness to suit your needs. xrandr --output <monitor-name> --brightness 0.7 Now that your eyes are comfortable, let me show