This story of summer started in a dull grey winter at home. Bored, I started lingering around IRC channels and much like the Alice of Wonderland, stumbled into the wonderful world of FOSS. Little did I know, it was gonna be one of the best things that happened to me. Below is a story of bugs, PRs , repos, commits, and some more commits. But it is also a story of curiosity, learning, frustrations (a lot of it), resilience (more than you think) and some amazing amazing people of the community. If I have to sum it up for you, I couldn’t think of a way better than this. So here it goes..
Disclaimer: It was a long journey, hence the long blog.
I started out slow. Figuring things out. Picard has a nicely organized and pretty code-base in python. Something that I have always admired and what motivated me to contribute to this project. I went through the tickets and found something I figured I could fix. It took me a complete day to figure my head around the code. Although there was a time when I felt helpless and was about to lose my patience, I was finally able to find where the logic broke and fix it. It felt amazing making my first contribution to Picard.
#7 PICARD-546,287: Add support for tag removal
My first big pull request, where I was actually implementing a feature rather than fixing a bug. The entire pull request took a week to complete including the code reviews. I had to delve and add code across the file and tag handling module of Picard.
#120 Return empty metadata instead of empty dictionary
It was finally a month since I had started contributing to Picard. One of my biggest motivations to contributing regularly was this post by our org admin ruaok before the start of GSoC '15. I wanted to hold myself to the same standard, and so I did, 1 PR a day. I was quickly fixing up as many tickets as I could, making sure I kept my mentor, zas' hands full with code reviews. Everyday was exciting for me as I would skim through Jira looking for tickets I could fix for the day. I managed to fix over 20 tickets in my first month.
#228 Add Picard 1.4 release notes
It was Valentine's Day and man o' man was it memorable day for my code-valentine - Picard. We finally released a major Picard version after more than 2 years and over 1000 commits. I also got featured on the release blog-post, something that made me feel really proud of all the work I had done so far. This was also the time that I had started to visualize how my GSoC project would look like. I started pitching proposals of a v2 for Picard and I got a lot of support for the direction that I wanted to head in. If you want a detailed look into what my final GSoC proposal looked like, click here.
#249 Merge pull request #632 from lalinsky/master
A few days, later another surprise awaited me - I finally got push access to the main Picard repo. This was the first pull request I merged. This pull request along with PR #633 was my first time foraying into the world of cross-platform issues. This I believe is the first OSX related bug I fixed, with help from luks and his trusty ol' Mac. After hours of remote sessions on the Mac that luks had setup for me, we were finally able to resolve a nasty bug that was affecting the OSX build.
#279 Update Qt5 port
This was around the time that I had started to work according to my proposal. We had already released Picard 1.4.1 to fix day 0 bugs that came with 1.4. The development efforts had started to shift toward Picard 2.0.
The first major GSoC task was up - Porting the entire code-base to Qt5.
It took a mammoth of a pull-request and multiple subsequent patches to finally have Picard up and running on Qt5. Major challenges to this port was the lack of documentation on porting for PyQt5. Apart from the PyQt porting guide, I also had to rely on the Qt5 porting guide written for C++ and figure out the equivalent python bindings. Lucky for me, the guys over at Riverbank (the people who made the PyQt bindings) have done a wonderful job at keeping the bindings as close to the C++ API as possible. This along with stack-overflow and the wonderfully documented Qt5 code, made the porting job much much easier.
Another challenge that I faced was making sure that the above code passed tests on our CI. Picard uses Travis CI, which uses Ubuntu 14.04 containers to build the code. Unfortunately, 14.04 does not have the PyQt5 package for Py2 and building it from source takes forever. As a result, I had to dockerize the repo and run the tests in Ubuntu 16.04 in docker in a Trusty container over at Travis CI. Talk about code-ception.
#280 Replace deprecated __builtin__
Next task on my GSoC checklist was equally big. I had to port all of the code to Py3. I was targeting Python 3.5 for the port. Of much help to me this time were the python tools, pylint and python-modernize. I used them to figure out the pieces of Py2 code that had been deprecated and to replace them with equivalent Py3 code. Major part of the problems I faced here were encoding issues. And believe me, they only get magnified if your app is cross-platform and multi-lingual.
Couple the above with PyQt5 behaving slightly differently for Py3, I had to plow through a lot of encoding/decoding/string/bytes issues. Pile Mutagen (a python audio tagging library that Picard depends on) on top of this mess and you have a recipe for an encoding-decoding disaster.
After lots of string/bytes fixes and banging my head against the wall porting this black magic code, I finally managed to port Picard to Py3 with all tests passing \o/.
You can find all the above commits in this massive PR.
#382 Update setup script to allow submitting pip packages
Next, to weed out further bugs, I wanted as many alpha testers as possible. To make their task easier, I packaged Picard into a pip package and released it on PyPi.
#393 PICARD-817: Enable high DPI support for Picard
I further tackled the issue of Picard not rendering properly on High-DPI screens. This issue mainly affected our OSX users and with the port to Qt5, it was easy to alleviate. Although not mentioned explicitly in the Qt5 documentation, I was able to find a couple of blog posts that helped me successfully implement it. Basically I needed to add up-scaled versions of the icons used by Picard in the Qt resources with a special suffix and Qt took care of the rest.
Since the upscaled version of the icons used by Picard were missing, I ended up designing the entire icon-set for Picard myself. This was my first time with tweaking with InkScape, but I think I managed fine. You can find the new icons here.
#404 Add coverage reports for codacy
Part of my GSoC project included testing the Picard code extensively. To help me get on with it, I setup Codacy on the Picard repo to display coverage and linting issues for all builds. This really helped with reviewing code and keeping contributions to the Picard repo clean and upto the quality that is expected. Coverage reports also allowed us to figure out which key pieces of Picard code was untested and target them to ensure there were as less regressions as possible.
#442 Migrate code to json API
My next big challenge was moving the webservice API calls from XML to JSON. Initally it meant refactoring the spaghetti code that was the Picard webservice module. I untangled the code and split it into an easy to add upon WebService and APIHelper classes.
Next came the actual porting part. I had to manually figure out the differences in the JSON and XML outputs of all the APIs used by Picard (mainly MusicBrainz API and AcoustID API).
This port also helped in figuring out the parts of MB JSON API that was missing or behaving differently than the XML API. Bitmap, our resident MB developer was quick to fix all the issues blocking the transition JSON API for Picard. Picard switching to the JSON API also served as a good way to test our upcoming MB WebService 2.5, that is supposed to be JSON only.
#485 Add appveyor config for windows tests
Testing a cross-platform app means you need to get your hands dirty on all 3 OSes. Primarily a Linux user, I had to blindly write CI configs for Windows and OSX. For Windows testing, I have not yet found a better platform than AppVeyor. It is free for FOSS projects and down-right amazing in terms of queuing and build speeds.
I didn't have any options when it came to OSX builds. Although Travis does provide the functionality, the number of OSX containers available are scarce compared to the number of jobs that get submitted. As a result, I faced long queuing times which made the blindly-writing-configs-and-using-Travis-to-test part very difficult. I managed to do it with a wee-bit of patience and the fact the OSX has bash.
#500 Add v2 plugin manifest compatibility
All was well and I was finally on the last step of my GSoC project. All that remained was tidying up the code, fixing any remaining bugs and switching to a more robust Plugin packaging and distribution API. I wrote a CLI tool for the same, which you can find here.
You can find all the above commits here.
Frankly, I would've never expected last year that my GitHub would look so green. I have made over 1200+ contributions this year to FOSS. My coding skills have taken a giant leap from last year. I would recommend every coder to try and contribute to a large open-source project at least once during their lifetime. It teaches you a lot of things that can frankly never be taught anywhere else, and the entire experience is totally worth it!
GSoC is a marvellous platform to incubate contributors to the open-source world. I want to thank my mentor zas (Laurent Monin), my org admin ruaok (Robert Kaye) for helping me throughout and making sure I had an amazing GSoC experience, and Google for making this possible.
This has been an amazing opportunity for me. MetaBrainz community has been nothing short of wonderful to me and I am looking forward to contributing more to MetaBrainz and exploring other FOSS projects.
Disclaimer: It was a long journey, hence the long blog.
The Story
#1 PICARD-851: Correctly update parent items on saveI started out slow. Figuring things out. Picard has a nicely organized and pretty code-base in python. Something that I have always admired and what motivated me to contribute to this project. I went through the tickets and found something I figured I could fix. It took me a complete day to figure my head around the code. Although there was a time when I felt helpless and was about to lose my patience, I was finally able to find where the logic broke and fix it. It felt amazing making my first contribution to Picard.
#7 PICARD-546,287: Add support for tag removal
My first big pull request, where I was actually implementing a feature rather than fixing a bug. The entire pull request took a week to complete including the code reviews. I had to delve and add code across the file and tag handling module of Picard.
#120 Return empty metadata instead of empty dictionary
It was finally a month since I had started contributing to Picard. One of my biggest motivations to contributing regularly was this post by our org admin ruaok before the start of GSoC '15. I wanted to hold myself to the same standard, and so I did, 1 PR a day. I was quickly fixing up as many tickets as I could, making sure I kept my mentor, zas' hands full with code reviews. Everyday was exciting for me as I would skim through Jira looking for tickets I could fix for the day. I managed to fix over 20 tickets in my first month.
#228 Add Picard 1.4 release notes
It was Valentine's Day and man o' man was it memorable day for my code-valentine - Picard. We finally released a major Picard version after more than 2 years and over 1000 commits. I also got featured on the release blog-post, something that made me feel really proud of all the work I had done so far. This was also the time that I had started to visualize how my GSoC project would look like. I started pitching proposals of a v2 for Picard and I got a lot of support for the direction that I wanted to head in. If you want a detailed look into what my final GSoC proposal looked like, click here.
#249 Merge pull request #632 from lalinsky/master
A few days, later another surprise awaited me - I finally got push access to the main Picard repo. This was the first pull request I merged. This pull request along with PR #633 was my first time foraying into the world of cross-platform issues. This I believe is the first OSX related bug I fixed, with help from luks and his trusty ol' Mac. After hours of remote sessions on the Mac that luks had setup for me, we were finally able to resolve a nasty bug that was affecting the OSX build.
#279 Update Qt5 port
This was around the time that I had started to work according to my proposal. We had already released Picard 1.4.1 to fix day 0 bugs that came with 1.4. The development efforts had started to shift toward Picard 2.0.
The first major GSoC task was up - Porting the entire code-base to Qt5.
It took a mammoth of a pull-request and multiple subsequent patches to finally have Picard up and running on Qt5. Major challenges to this port was the lack of documentation on porting for PyQt5. Apart from the PyQt porting guide, I also had to rely on the Qt5 porting guide written for C++ and figure out the equivalent python bindings. Lucky for me, the guys over at Riverbank (the people who made the PyQt bindings) have done a wonderful job at keeping the bindings as close to the C++ API as possible. This along with stack-overflow and the wonderfully documented Qt5 code, made the porting job much much easier.
Another challenge that I faced was making sure that the above code passed tests on our CI. Picard uses Travis CI, which uses Ubuntu 14.04 containers to build the code. Unfortunately, 14.04 does not have the PyQt5 package for Py2 and building it from source takes forever. As a result, I had to dockerize the repo and run the tests in Ubuntu 16.04 in docker in a Trusty container over at Travis CI. Talk about code-ception.
#280 Replace deprecated __builtin__
Next task on my GSoC checklist was equally big. I had to port all of the code to Py3. I was targeting Python 3.5 for the port. Of much help to me this time were the python tools, pylint and python-modernize. I used them to figure out the pieces of Py2 code that had been deprecated and to replace them with equivalent Py3 code. Major part of the problems I faced here were encoding issues. And believe me, they only get magnified if your app is cross-platform and multi-lingual.
Couple the above with PyQt5 behaving slightly differently for Py3, I had to plow through a lot of encoding/decoding/string/bytes issues. Pile Mutagen (a python audio tagging library that Picard depends on) on top of this mess and you have a recipe for an encoding-decoding disaster.
After lots of string/bytes fixes and banging my head against the wall porting this black magic code, I finally managed to port Picard to Py3 with all tests passing \o/.
You can find all the above commits in this massive PR.
#382 Update setup script to allow submitting pip packages
Next, to weed out further bugs, I wanted as many alpha testers as possible. To make their task easier, I packaged Picard into a pip package and released it on PyPi.
#393 PICARD-817: Enable high DPI support for Picard
I further tackled the issue of Picard not rendering properly on High-DPI screens. This issue mainly affected our OSX users and with the port to Qt5, it was easy to alleviate. Although not mentioned explicitly in the Qt5 documentation, I was able to find a couple of blog posts that helped me successfully implement it. Basically I needed to add up-scaled versions of the icons used by Picard in the Qt resources with a special suffix and Qt took care of the rest.
Since the upscaled version of the icons used by Picard were missing, I ended up designing the entire icon-set for Picard myself. This was my first time with tweaking with InkScape, but I think I managed fine. You can find the new icons here.
#404 Add coverage reports for codacy
Part of my GSoC project included testing the Picard code extensively. To help me get on with it, I setup Codacy on the Picard repo to display coverage and linting issues for all builds. This really helped with reviewing code and keeping contributions to the Picard repo clean and upto the quality that is expected. Coverage reports also allowed us to figure out which key pieces of Picard code was untested and target them to ensure there were as less regressions as possible.
#442 Migrate code to json API
My next big challenge was moving the webservice API calls from XML to JSON. Initally it meant refactoring the spaghetti code that was the Picard webservice module. I untangled the code and split it into an easy to add upon WebService and APIHelper classes.
Next came the actual porting part. I had to manually figure out the differences in the JSON and XML outputs of all the APIs used by Picard (mainly MusicBrainz API and AcoustID API).
This port also helped in figuring out the parts of MB JSON API that was missing or behaving differently than the XML API. Bitmap, our resident MB developer was quick to fix all the issues blocking the transition JSON API for Picard. Picard switching to the JSON API also served as a good way to test our upcoming MB WebService 2.5, that is supposed to be JSON only.
#485 Add appveyor config for windows tests
Testing a cross-platform app means you need to get your hands dirty on all 3 OSes. Primarily a Linux user, I had to blindly write CI configs for Windows and OSX. For Windows testing, I have not yet found a better platform than AppVeyor. It is free for FOSS projects and down-right amazing in terms of queuing and build speeds.
I didn't have any options when it came to OSX builds. Although Travis does provide the functionality, the number of OSX containers available are scarce compared to the number of jobs that get submitted. As a result, I faced long queuing times which made the blindly-writing-configs-and-using-Travis-to-test part very difficult. I managed to do it with a wee-bit of patience and the fact the OSX has bash.
#500 Add v2 plugin manifest compatibility
All was well and I was finally on the last step of my GSoC project. All that remained was tidying up the code, fixing any remaining bugs and switching to a more robust Plugin packaging and distribution API. I wrote a CLI tool for the same, which you can find here.
You can find all the above commits here.
Afterthoughts:
By some stroke of fate, my total commits to the Picard repo are exactly 500 in number as of this post. I shook my head in disbelief and amazement as I let it sink in. I have come a long long way. During this entire journey, I have learnt a lot of new things: version control, making cross-platform desktop apps, proper unit-testing, i18n, CI, making CLI tools, designing APIs, documenting and reviewing code and most of all writing modular, manageable and maintainable code.
Expectation
Reality
Frankly, I would've never expected last year that my GitHub would look so green. I have made over 1200+ contributions this year to FOSS. My coding skills have taken a giant leap from last year. I would recommend every coder to try and contribute to a large open-source project at least once during their lifetime. It teaches you a lot of things that can frankly never be taught anywhere else, and the entire experience is totally worth it!
GSoC is a marvellous platform to incubate contributors to the open-source world. I want to thank my mentor zas (Laurent Monin), my org admin ruaok (Robert Kaye) for helping me throughout and making sure I had an amazing GSoC experience, and Google for making this possible.
This has been an amazing opportunity for me. MetaBrainz community has been nothing short of wonderful to me and I am looking forward to contributing more to MetaBrainz and exploring other FOSS projects.