GSoC-Week7

This week resulted in some PRs -  for instance IRSA is now merged as is the PR for UKIDSS. Apart from that there are also two ongoing PRs one for NRAO and the other for NIST.

On a side note, one major PR that was merged this week is astropy#1006 so that now astropy.coordinates.Angle is now a subclass of astropy.units.Quantity. This also resulted in some changed in astroquery that uses the Angle class extensively - astroquery now works with astropy versions >= 0.3.

One git fun fact I learnt this week was about how to clean a git directory when building inline which is what I usually do -  well I ran into some errors because of this while installing astropy but this was cleared up thanks to the extremely helpful discussion here.

Another interesting thing I learnt was about automatically generating functions in python - this can be quite helpful in cases where two functions differ only slightly and can significantly reduce the amount of redundant code.  Since quite a lot of our code currently uses such functions, my mentor made a PR that does just this.

Cheers,
Madhura!

GSoC-Week6

The PR for NED is now merged and the ongoing PR right now is for IRSA. Details of this can be seen here. What still remains in this is to add an interface to ATLAS, which I described in my previous post.

Along with this I am also fiddling with the UKIDSS module that is right next on the refactoring TODO, though I am yet to make a PR for this yet.

One interesting titbit I had like to share is about editing commits that you have already pushed to Git.For instance say you committed some changes and then pushed these, but suddenly realized that you had forgotten to do a git add for some files that also went with the commit. Correcting this is very easy since the commit you would like to change is the last commit. First simply add whatever changes you wished had gone along with the commit. Now do:

git commit --amend

And all these changes will be added as a part of the commit as well. Now simply force push (git push -f )this commit back to Git

And lo! there will be a shiny new commit with all the changes as desired. Since GSoC is my first brush with Git, it feels very nice to discover neat facts like this!

Cheers till next week!

GSoC-Week5

Time flies! As this post marks the fifth(!) week of GSoC, I can't help feeling how true this (albeit hackneyed) saying is! So here goes the progress for this week:

This week resulted in one more PR with the NED web service. There are finishing touches to be added and docs to be done but apart from that the PR can now be seen here.

Since Astroquery is basically about querying web services, it involves writing methods to fetch various astronomical data such as spectra, images, astronomical references, etc. Most of these queries require similar arguments such as the object we are interested in querying, in addition to some query specific arguments that are relevant only to a particular kind of query. So what I had been doing is write a separate method for each of these queries. This led to quite some repetition also blowing up the code base. My mentor suggested that a better way to achieve the same thing would be to write a common method for all these queries, with optional keyword args that were relevant to specific query types. So for instance initially while I had

def get_positions(*args, **kwargs):
    # code

def get_diameters(*args, **kwargs):
    #code

...

after the refactor the effect was now similar to

def get_table(object_name, table='positions', **kwargs):
    #code

So now all the common arguments are abstracted out, and the optional specific to each query type just go into **kwargs. Amazingly this cut down the code by 455 sloc at the cost of adding 129 sloc. Awesome!. The specific commit pertaining to this is here

The next module coming up is IRSA. Of course most of this is tidying up converting functions to class methods, etc... Also we'll be adding an interface to the ATLAS service which is available under IRSA. This should add functionality to access several image services, apart from the NED images...

PS I am now using Syntax Highlighter for code snippets since gists may be troublesome for RSS readers.

GSoC - Week4

Hi! - I am a day late this time I guess. So this week I have mainly been finalizing Vizier. I have also worked on some code for the next module i.e. NED... though its a WIP and I am yet to make a PR.

Apart from this there were a couple of other auxilliary commits this week mainly to make work easy for the mainstream PRs. So there were PRs - #126, #127 and #128.

One very useful thing I learnt during these PRs was rebasing a git branch. Basically there were some conflicting commits and so the branch couldn't be automatically merged in the master. So I needed to rebase - resolve those conflicts and force update my PR's ( had to do this with two of the PRs!). I hadn't much familiarity with rebasing but astropy's documentation came out to be very handy. One interesting tool that I also learnt to work with is Meld. Meld is a visual tool that can give a huge helping hand in those 3 way merges - you have the local file, the remote file and the conflicting file that you must fix. With Meld all these files and the differences between them are visualized in one window - with simple clicks you can decide the diff you want to go with and lo presto you are done!

Another quite unrelated but very useful thing I found is that IPython has this lovely feature in which you can just feed in code with all the leading prompts(>>>) and (...) (for instance if you are copying from some docs, etc) and you don't need to do any editing, IPython understands that this is code meant to be executed. I realize (after reading this again) that I may not sound very obvious so read this up at the official IPython docs. It is a real time saver!

So Long :)

GSoC - Week3

So this week I have mainly been wrapping up Simbad as well as pushing in some preliminary code for VizieR. Both irsa_dust and Simbad are merged now. The PR for VizieR  is also up.

So the upcoming week should mainly involve completing Vizier and dealing with some more modules.

There's nothing much that I need to share this week - so this it for now!

UPDATE: Here are some working examples of Vizier in an IPython Notebook, browse them on nbviewer!

GSoC - week2

So this week,  we have started looking at another module SIMBAD.  Of course there is only preliminary discussion/ review yet but the code skeleton is available in this PR.

One issue I am working on goes like this - What we would like to do is display an HTML table (this one) to the user - to let her decide the VOTable fields to set. The first step is to scrape the table using BeautifulSoup(bs4). Next we'll be storing the results as JSON - again with python this is as simple as a couple of lines of code. What I am looking at is a great way to then display the data interactively in python/ipython.

The obvious way to do this would be to use something like a table. Good News - Astropy already has this functionality and it is *truly fully loaded*. With astropy.table.Table - you can scroll the rows (replicate the unix more) and in an IPython NB, the table is displayed as an HTML table and looks very impressive. I spent quite a few hours poring over the source code and I was really inspired :)!

Well then I tried my hand at adding rows/columns and creating a table from the HTML I had scrapped and I came across a funny unicode error. This was just a false alarm :) - since an astropy table uses a numpy array underneath - in a numpy array if you need to enter a string of variable length you should also mention a datatype for it otherwise it tries to parse this by its default datatype and then errors ensue! (This then lead to a tour of numpy datatypes which in turn lead to an enjoyable diversion to the Scipy-2013 web page - and I spent a few minutes(?) looking excitedly at the plans - Astropy is presenting there(and its at UT-Austin too!) ) Well so what is needed is dtypes=['object'] for a variable length string.  Well but there are some points to be smoothed out yet - which is one of the things I am looking at.

P.S. The code I include via a gist doesn't appear in the Feeder (this post doesn't have any but the June-16th one did) - looking into this.

GSoC - Week-1

This week was mainly spent in refactoring the astroquery.irsa_dust module. A detailed discussion of the entire implementation can be found at this pull-request.  And as it currently stands most of the API for this is almost finalized except for some tweaks that may come up in the final review.

Apart from this the order in which the remaining modules of astroquery will be refactored may be found here.

Some other things that were also tackled this week were to monkeypatch the remote tests that most commonly resulted in network errors/ timeouts. Again more details here.

Currently SIMBAD is on the top of our to-do list. So I have been trying to familiarize myself with this web service and also made some preliminary attempts at re-writing the code, coherent with the finalized API.

P.S. Anyone with an Astronomical incline, could probably fork the revised irsa_dust module , use it and give feedback!

GSoC - Coding period starts!

The GSoC coding period starts from 17th June (today!). The last week has been quite enjoyable. The best part of GSoC is that it offers you the opportunity to learn many many new and useful things. This week I have mostly been trying my hand at refactoring the irsa_dust module in astroquery as per the API finalized last week.

One very useful software that I had mainly heard about but never used myself is the IPython Notebook. It is great for creating examples to demonstrate functionality and can be shared using gists and nbviewer It has very handy features like auto-completion - and I am glad that I have it installed and running now!

Another very useful thing I learnt is testing in python. Astropy uses pytest as its main testing framework. It is very simple to write tests in this. In-fact its catch line is No API!. Given its superb test discovery rules and features - it sure is fun to get started writing tests using pytest.

One very useful trick my mentor showed me is to use the @pytest.mark.parametrize decorator. This is very useful if say you want to test a function for several different values of a parameter(s). Rather than writing redundant tests you could do something like :

This will then test the funtion for each value of the parameters coordinate and radius and check that the output is as expected.

Another feature that I also learnt about is monkeypatching. This can be used to create dummy objects, that can mimic the behavior of a real resource - for instance HTTP response and request objects. This can be used in astroquery to create patches that replace actual responses/requests to the remote server.

Also this week I received  a one-year student membership to ACM (that includes a subscription to the digital library) as a part of GSoC!

GSoC - week 2 of interim period

This week I have mainly been reading up the Astroquery code base as well as relevant portions of Astropy. I also looked up on some advanced Python techniques like decorators, descriptors, generators and context managers and OOP done via Python. I found some really helpful resources on the web, I will just list them here:

This is definitely a must read:
http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html

http://www.pgbovine.net/python-idioms.htm

This may require a sign in but it has some really good links:
https://www.udacity.com/wiki/cs212/python_learning_aids

http://wiki.python.org/moin/PythonDecoratorLibrary#Property_Definition

And also an ebook:
http://www.linuxtopia.org/online_books/programming_books/python_programming/

Finally the best compilation of all python aspects at one place - The Hitchhiker's  guide to Python!:
http://docs.python-guide.org/en/latest/

Also the API for Astroquery is almost finalized! And the revised version may be found here:
http://astroquery.readthedocs.org/en/latest/astroquery/api.html

Thats all for now!

Astropy : Getting started

The community bonding period of GSoC lasts till 17th June. This is a great opportunity to get familiar with your organization and the repo to which you will be contributing. So far I have set up my work environment. The Python programming I did so far was never a part of an entire software project , so I have been configuring my environment to make this process efficient.

The two sources that have been a huge help to me starting from the application period are:

http://docs.astropy.org/en/latest/development/codeguide.html

http://docs.astropy.org/en/latest/development/workflow/development_workflow.html

They provide very clear step by step guidelines of contributing to Astropy.

Also I have configured my emacs as per the directions given  in http://docs.astropy.org/en/latest/development/codeguide_emacs.html
This makes it very easy to maintain the required indents, pep8 guidelines, etc when developing with emacs. It is very much a must-do. Astropy has also generously provided me with the license to a full-fledged commercial Python IDE - PyCharm. So I am also trying my hand at that.

Another very useful suggestion I got from Astropy  is to become acquainted with virtualenv. So I have also created a virtualenv for Astropy/Astroquery on my system.

Besides this I have been reading up on a few topics :

Firstly I have been looking up the PEP 8 style guide for Python.

Also since we will be documenting everything with Sphinx, I found a nice tutorial that gave me a good introduction to this.

I had already forked the repos for astroquery and astropy. So now I am  getting familiar with the codebase.

Apart from this there is a growing list of Tasks (thanks org-mode!) that I intend to finish in the community bonding period.

Looking forward to an exciting summer!

The GSoC'13 results are out and my proposal to Astropy under the Python Software foundation is accepted! Through this summer I will be working on astroquery so that its first stable version can be released. Astroquery is an affiliated package of Astropy aimed to make it easy for Astronomers to access online data sources like ViZier, SIMBAD, IRSA and others, from Python programs.

I will be updating this blog weekly as we progress through this summer. Your suggestions for this project are welcome to the astropy-dev mailing list.

GSoC'13 and the exciting world of Open Source

Applications for the Google Summer of Code are now open! I have always been intending to apply for GSoC since I heard of it probably last year. Unfortunately though last year - I just couldn't spare the time to apply to GSoC. I really am regretting this now. The application and pre-application phase of GSoC in itself provide an amazing learning experience.

I have relied heavily on Open Source for all my Computer Science needs.  Emacs, TeX,  Unix, ..., all the tools I am so used to are thanks to legends like Richard Stallman, Knuth,  Dennis Ritchie and the remarkable Open Source vision.  Every time I log into my Linux system, I can't help looking upto OSS for giving me the flexibility and freedom to get my things done with such ease and style. Truly the best things in the world are free. And while I have often thought of making contributions big or small to Open Source, I never really got around to it till GSoC'13 arrived. This time round I had really made up my mind to apply.

...And I am really glad that I did. There are 177 mentoring organizations in participating in this year's GSoC - thus offering lots of diversity to probably satisfy anyone's interests. After scouring the list for long hours, I came across Astropy under the Python Software Foundation. I have always wanted to explore Astronomy while also building up on my programming skills and applying here presented me with the wonderful opportunity to get the best of both worlds this summer! Thus began my application period filled with goodies : I joined mailing lists, idled on the IRC but most importantly I created a git-hub account.  I learnt to fork a repository, work on a patch and open a pull request. It was also immensely satisfying when my patch was finally merged.

I can't thank GSoC enough for introducing me to the exciting world of open source.  I am really itching to make more contributions, I also found a couple of other interesting repos like NLTK. But with exams round the corner I will probably have to wait for a couple of weeks, before I dive deeper into the exciting world of open source. :-)

printf("Hello World!\n");

Some meanings first...

Vyom (Sanskrit : व्योम ) :  In Sanskrit Vyom means the infinite and endless space.
ping :  To know about this go to the shell and type   man ping  :-)

*   *   *

I am a Computer Science student, fascinated by Astronomy... This blog is about my experiences and explorations in these two remarkable fields.