Skip to main content

Recap: SciPy 2015 - Synopsis

SciPy 2015 has come and gone. If I step back, what are some of the learning lessons?

There were certain themes that recurred from talk to talk:

  1. Speed. One of the perceived limitations of Python seems to be speed of execution which is important to process very large datasets. Many talks dealt with this topic in various ways. Some of the approaches included enabling process parallelization (Dask, DistArray), GPU-acceleration (VisPy), or acceleration via some means of compilation - sometimes just-in-times (Numba). With these tools, Python is no longer slow. It's impressive that the combination of these approaches has enabled data scientists to process 60+ GB data sets as if they would be loaded into memory on one small laptop that actually only has 8-16 GB of memory.
  2. Visualization was a theme. So many talks dealt with making complex data sets visible, and they did so to address different issues: Serialization to enable interactivity (Bokeh, matplotlib), visualizing large dynamic datasets with low latency (VisPy), making it easier for normal scientists to view and share visual graphs (Holoviews).
  3. Reproducibility and transparency of data analysis in science which includes the concept of literate programming was a theme. Again, different solutions were presented. The solutions ranged from ensuring that data analysis was performed on the same configurations on different levels (Docker, Dexter), to striking the right balance between enough metadata while also having beautiful formatting, to reducing error rates by automating execution of different simulation scenarios.
  4. The scientific Python ecosystem is developing at the speed of light. Traditionally, R has been said to be superior to Python when it comes to availability of specialty packages that could do one specific thing. Well, I heard again and again that this difference is rapidly disappearing. And because Python is a more universal programing language, there are things Python code can do that R code cannot. Many quality packages are developed at rapid speed. Just take Bokeh for instance. Last year: 0.5. This year, the module will reach production quality with the intended release of V1.0 later this year. And this is just one example.
  5. Having pointed out the previous point, I will say as well that the language wars are over. Another fascinating theme was going post-language wars. Instead of trying to convince X, that language Y is the best and all else sucks, there is a desire to go beyond that. Bokeh, for example, now has bindings for R and Julia. The evolved Ipython Notebook, now called Jupyter, which actually stands for Julia, Python and R, is now language agnostic. The individual language capabilities are supposed to be added via loading specific kernels. Currently, the most active kernel developments are in the R and Julia space. There is also discussion about a universal data storage structure that could be shared at least between the 3 previously mentioned languages. These are thoughts that have started to be seriously discussed in the scientific Python community. Scientific Python developers have started reaching out to the R and Julia community but these discussions are very nascent.
  6. Python and teaching was a theme. Ipython notebook and the ecosystem developing around that make great ways to teach students science, programing, etc.
  7. Develop in Python 3! Last but not least, in the past, Python 3 migration and development has always hinged on the availability of Python 3 compatible modules. Even last year this was still true. This year, there was universal consensus that Python 3 is no longer the future. The time to migrate is now!
In closing, for me this conference was easily the most exciting conference I have ever attended.

It can be said that the organizers outdid themselves in putting together a conference so well. As mentioned earlier within just the last 3 years, the conference has grown from 300 to more than 600 conference attendees. SciPy is no longer small. But despite the fact, the conference has successfully retained an atmosphere of a small family meeting. The growth in conference attendance at the least shows the increased interest in using Python as a language to perform all kinds of data science computation.

Beyond this though, the combination of high quality talks, and balance with social interactions has made this my default conference to go to over the next few years! See you all in 2016!

Popular posts from this blog

Focus on Algae - Part II: Energy

In the last focus section, we discussed how algae can be used to treat waste waters and mitigate CO2 in the process. Today's post will explore how algae can be used for energy generation. As already mentioned in the last time, biofuels have become very visible as of late due to environmental, economical and geopolitcal reasons. If at the heart of traditional biofuel generation lies in the creation and decomposition of biomass, then it would be easy to substitute corn or other less controversial land-based plants with algae. Although a lot of attention is paid to the use of algae in biofuel generation, and this article also mainly focusses on this aspect, it should be noted that algae can also be used to generate electricity by direct combustion of the biomass. Plans for these kinds of schemes are already on the way in Venice and a few other European locations [1].

Algae and Biofuels

What happens to the biomass after it has been created depends on the type of biofuel that is desired…

Sustainable Living: One man's trash...

Since Earth Week is starting tomorrow, I wanted share with you some concrete ways of how individuals like you and me can make an impact on a wider scale. I then also wanted to use this example to challenge everyone to think creatively about the larger context.

So you know how the saying goes: "One man's trash is another one's treasure." Today, I want to talk to you about garbage. Plastic garbage specifically. Plastic is quite a wondrous material. Made from oil by man with just a few additives can turn this polymer into so many different sorts of plastics with so many different properties from thin and flimsy plastic bags, to the carpet on which I am standing, to this plastic bottle from which I am drinking.

Focus on Algae - Part I: Bioremediation

After spending the last few blog posts on different aspects of dissimilatory bacteria, I want to switch the focus to a different class of organisms I have been interested in for a long time now. These are the algae. Algae comprise a large diversity of "sea weeds" and an even larger variety of single-celled organisms that mostly are capable of doing photosynthesis. They include the ordinary sea-weed, and make up a portion of the green slime found around the edges and the bottom of a pond. More exotic types of algae can live symbiotically - that is together with another organism in a mutually beneficial way. Lichens are an example of symbiotic relationship between algae and fungi. More information about the evolution and lineage of algae can be found in this wiki article.
Image via Wikipedia
Typically, these organisms are either not mentioned at all or only in conjunction with toxic algal blooms. But lately, algae, of course, have been in the news recently because of the promi…