I have been using MATLAB for eight good years. MATLAB served and still serves my purpose beautifully. In this entire span of time, I made repeated attempt to switch my go-to language from MATLAB to other alternatives, like Python or R, several times. The reason has never been that my work require something that is not being offered by MATLAB. The reason mainly is: MATLAB is a proprietary environment. It is an expensive piece of software. This is a huge turn-down for, at least, small businesses. The amount of cost it requires for licenses and its toolboxes is only justifiable by organizations of monstrous scale, like universities and giant corporations. In terms of transferability of skills, having expertise over MATLAB is getting into more niche region over the years.
However, all of these attempts of switching were failed. The reasons are following. I had been getting exhausted spending days to set up an environment in Python to solve a problem that takes 30 minutes altogether to solve. I prefer writing algorithms from scratch since I am more into theoretical aspects due to the nature of my research. It gives me a better functional understanding of what I am doing. Every time I opted for writing my own code instead of using in-built functions and modules that come as a part of packages, I was overwhelmed by its huge execution time. The counter-intuitive (in my opinion) approach offered by some popular packages (I must mention Pandas for Python) forced me to go through stackoverflow before writing every single line. Plotting the results had been a nightmare in its own rights. All of these bottlenecks made my transition from proprietary to open-source paradigm really harsh and unwelcoming. I guess I am not alone here, moreover, I believe I represent a modestly sized community in research computing.
After going through extensive reviews and research, I have finally set myself up to make a comprehensive attempt to switch my go-to language from MATLAB to Julia. But before that, I would like to point out some mistakes from my end which made my previous attempts unsuccessful. All of the hurdles I mentioned earlier are not at all the downsides of open-source programming languages, rather these are the parts of the features which enabled these languages to play central roles in many developments.
Mistake #1: Not using/appreciating virtual environment
If I am to rank the mistakes I did while switching, this will earn the trophy for sure. While there are tons of articles about ins and outs of virtual environment in Medium, only a few address the actual role of virtual environment in research computing. I learned it in a hard way after massacring my Linux distribution and I am glad I did. Let me give an example. The following example is valid for Python, R, Julia or other open-source languages.
Let us say, Shiblu built a package called “A”. Using several features of the package A, Folu built a package called “B”. As a matter of fact, some dependencies are introduced between A and B. Later Shiblu decided to take an entirely different approach, modified A and released a new version called A2. But Folu moved on and got busy with other things, forgetting to update B. When a user updates his/her packages and end up updating A2 in process, he/she essentially ends up crippling the functionality of B in that system. This is exactly what happened to me and it threw me in nasty quagmires several times.
The scenario described above is extremely common in open-source paradigm owing to its heavily decentralized development cycles. To address this issue, virtual environment exists which lets you instantiate different instances and let you different versions of packages in a single system. Not only that, it helps you to reproduce your codes since these environments can be exported. So, one of the very first things I learned in Julia is- how can I activate, instantiate and export virtual environments?
Mistake #2: “I just need to pick up the syntax” syndrome
If you switch from proprietary programming language to open-source ones, this particular syndrome will surely be a deadly pitfall. Let me elaborate why I call this a syndrome since I had been a victim of it. I used to think that I will master Python within a few hours because all I need to do is to pick up the syntax and practice a little.
Let us take Mathworks (creator of MATLAB) for example. It is extremely easy for them to maintain and coordinate a uniform practice of syntax designing because of its organizational hierarchy. As a result, you can move across various toolboxes and packages while riding along a smooth learning curve. This is not true for open-source languages at all. It is way beyond picking up the syntax because, that is not the end of the story. You literally need to pick up the cultures. Let me clarify. Different packages are built by different groups of developers. Naturally, they share widely varying ideology in terms of syntax design and that reflects on their packages as well. While I am sure that there are serious efforts to minimize the asymmetry in terms of syntax across various packages, it is not entirely possible to wipe it out. Hence, the uniformly intuitive syntax that exists in MATLAB can be incredibly rare in open-source languages. It may sound harsh but you are living in fools’ heaven if you expect so. I need to mention that, easy and intuitive are two different aspects in this context. Python is known for its easy human-esque syntax but in cases, at least in my experience, it can be counter-intuitive. Moreover, two different packages can share modules/functions with similar role. This can be a bottleneck in designing a standard framework of doing things.
For me, overcoming this seems a little tricky and it has to be done proactively. So far the strategy is, pick as few packages as possible which serves your purpose and master the hell out of them. Doing so will help me to develop/demonstrate a Proof of Concept (PoC) in short time and accelerate my problem solving time.
This is it for today. I will keep documenting my transitions. Cheers!
Shah Mahdi Hasan 