GSoC ends but the awesomeness doesn't!

So, I was re-visiting my blog entries from GSoC 2013 and realized that my blogs seemed to taper off quite suddenly. I guess that joining grad school brought along an entire whirlwind that blew other things out of my mind! But this is definitely not a fitting ending for what was far by my best summer so far.

So here goes. Since 2013, astroquery has participated in later GSoC's and made some really nice progress,  and Astropy has made some truly astronomical(!) progress as well. I am totally glad to be a a part of such a vibrant and active community of pythonistas!

I learnt so much over that summer, thanks too my extremely helpful,  jolly and awesome mentors ( Thanks keflavich and astrofrog!), and due to fantastic discussions on the mailing list.  I can only realize the immense value of my experience as I look back at my grad school and count the millionth time,  git saved my life or a python script saved me from becoming a sleep-deprived zombie.  All in all, I can't believe my luck that I got to work with one of the best orgs in GSoC.!

And writing all this is just making me realize how much I love contributing to open source.  Thanks to some really nice policies at my work, I now have the go ahead to contribute to open source. So hopefully I'll get started again!

*Disclaimer : all views are my own and not endorsed in any way by my employer.

GSoC-Week7

This week resulted in some PRs -  for instance IRSA is now merged as is the PR for UKIDSS. Apart from that there are also two ongoing PRs one for NRAO and the other for NIST.

On a side note, one major PR that was merged this week is astropy#1006 so that now astropy.coordinates.Angle is now a subclass of astropy.units.Quantity. This also resulted in some changed in astroquery that uses the Angle class extensively - astroquery now works with astropy versions >= 0.3.

One git fun fact I learnt this week was about how to clean a git directory when building inline which is what I usually do -  well I ran into some errors because of this while installing astropy but this was cleared up thanks to the extremely helpful discussion here.

Another interesting thing I learnt was about automatically generating functions in python - this can be quite helpful in cases where two functions differ only slightly and can significantly reduce the amount of redundant code.  Since quite a lot of our code currently uses such functions, my mentor made a PR that does just this.

Cheers,
Madhura!

GSoC-Week6

The PR for NED is now merged and the ongoing PR right now is for IRSA. Details of this can be seen here. What still remains in this is to add an interface to ATLAS, which I described in my previous post.

Along with this I am also fiddling with the UKIDSS module that is right next on the refactoring TODO, though I am yet to make a PR for this yet.

One interesting titbit I had like to share is about editing commits that you have already pushed to Git.For instance say you committed some changes and then pushed these, but suddenly realized that you had forgotten to do a git add for some files that also went with the commit. Correcting this is very easy since the commit you would like to change is the last commit. First simply add whatever changes you wished had gone along with the commit. Now do:

git commit --amend

And all these changes will be added as a part of the commit as well. Now simply force push (git push -f )this commit back to Git

And lo! there will be a shiny new commit with all the changes as desired. Since GSoC is my first brush with Git, it feels very nice to discover neat facts like this!

Cheers till next week!

GSoC-Week5

Time flies! As this post marks the fifth(!) week of GSoC, I can't help feeling how true this (albeit hackneyed) saying is! So here goes the progress for this week:

This week resulted in one more PR with the NED web service. There are finishing touches to be added and docs to be done but apart from that the PR can now be seen here.

Since Astroquery is basically about querying web services, it involves writing methods to fetch various astronomical data such as spectra, images, astronomical references, etc. Most of these queries require similar arguments such as the object we are interested in querying, in addition to some query specific arguments that are relevant only to a particular kind of query. So what I had been doing is write a separate method for each of these queries. This led to quite some repetition also blowing up the code base. My mentor suggested that a better way to achieve the same thing would be to write a common method for all these queries, with optional keyword args that were relevant to specific query types. So for instance initially while I had

def get_positions(*args, **kwargs):
    # code

def get_diameters(*args, **kwargs):
    #code

...

after the refactor the effect was now similar to

def get_table(object_name, table='positions', **kwargs):
    #code

So now all the common arguments are abstracted out, and the optional specific to each query type just go into **kwargs. Amazingly this cut down the code by 455 sloc at the cost of adding 129 sloc. Awesome!. The specific commit pertaining to this is here

The next module coming up is IRSA. Of course most of this is tidying up converting functions to class methods, etc... Also we'll be adding an interface to the ATLAS service which is available under IRSA. This should add functionality to access several image services, apart from the NED images...

PS I am now using Syntax Highlighter for code snippets since gists may be troublesome for RSS readers.

GSoC - Week4

Hi! - I am a day late this time I guess. So this week I have mainly been finalizing Vizier. I have also worked on some code for the next module i.e. NED... though its a WIP and I am yet to make a PR.

Apart from this there were a couple of other auxilliary commits this week mainly to make work easy for the mainstream PRs. So there were PRs - #126, #127 and #128.

One very useful thing I learnt during these PRs was rebasing a git branch. Basically there were some conflicting commits and so the branch couldn't be automatically merged in the master. So I needed to rebase - resolve those conflicts and force update my PR's ( had to do this with two of the PRs!). I hadn't much familiarity with rebasing but astropy's documentation came out to be very handy. One interesting tool that I also learnt to work with is Meld. Meld is a visual tool that can give a huge helping hand in those 3 way merges - you have the local file, the remote file and the conflicting file that you must fix. With Meld all these files and the differences between them are visualized in one window - with simple clicks you can decide the diff you want to go with and lo presto you are done!

Another quite unrelated but very useful thing I found is that IPython has this lovely feature in which you can just feed in code with all the leading prompts(>>>) and (...) (for instance if you are copying from some docs, etc) and you don't need to do any editing, IPython understands that this is code meant to be executed. I realize (after reading this again) that I may not sound very obvious so read this up at the official IPython docs. It is a real time saver!

So Long :)

GSoC - Week3

So this week I have mainly been wrapping up Simbad as well as pushing in some preliminary code for VizieR. Both irsa_dust and Simbad are merged now. The PR for VizieR  is also up.

So the upcoming week should mainly involve completing Vizier and dealing with some more modules.

There's nothing much that I need to share this week - so this it for now!

UPDATE: Here are some working examples of Vizier in an IPython Notebook, browse them on nbviewer!

GSoC - week2

So this week,  we have started looking at another module SIMBAD.  Of course there is only preliminary discussion/ review yet but the code skeleton is available in this PR.

One issue I am working on goes like this - What we would like to do is display an HTML table (this one) to the user - to let her decide the VOTable fields to set. The first step is to scrape the table using BeautifulSoup(bs4). Next we'll be storing the results as JSON - again with python this is as simple as a couple of lines of code. What I am looking at is a great way to then display the data interactively in python/ipython.

The obvious way to do this would be to use something like a table. Good News - Astropy already has this functionality and it is *truly fully loaded*. With astropy.table.Table - you can scroll the rows (replicate the unix more) and in an IPython NB, the table is displayed as an HTML table and looks very impressive. I spent quite a few hours poring over the source code and I was really inspired :)!

Well then I tried my hand at adding rows/columns and creating a table from the HTML I had scrapped and I came across a funny unicode error. This was just a false alarm :) - since an astropy table uses a numpy array underneath - in a numpy array if you need to enter a string of variable length you should also mention a datatype for it otherwise it tries to parse this by its default datatype and then errors ensue! (This then lead to a tour of numpy datatypes which in turn lead to an enjoyable diversion to the Scipy-2013 web page - and I spent a few minutes(?) looking excitedly at the plans - Astropy is presenting there(and its at UT-Austin too!) ) Well so what is needed is dtypes=['object'] for a variable length string.  Well but there are some points to be smoothed out yet - which is one of the things I am looking at.

P.S. The code I include via a gist doesn't appear in the Feeder (this post doesn't have any but the June-16th one did) - looking into this.