For large tables in R `dplyr's` function `inner_join()` is much faster than `merge()`

Posted on April 30, 2014 by zev@zevross.com

Using the merge() function in R on big tables can be time consuming. Luckily the join functions in the new package dplyr are much faster. The package offers four different joins: inner_join (similar to merge with all.x=F and all.y=F) left_join (similar to merge with all.x=T and all.y=F) semi_join (not really an equivalent in merge() unless […]

Continue reading →

R, Python, PostgreSQL (and more): A data science workflow example

Posted on April 29, 2014 by zev@zevross.com

Although many data science-related projects can be completed with a single software tool we often find that decisions about what tool to use for a project involve weighing a combination of what tool would be “best” for the job, what tools we're most familiar with and whether we already have scripts we can use. As […]

Continue reading →

Adding ArcGIS map services to your map using Leaflet

Posted on April 25, 2014 by zev@zevross.com

A lot of government agencies and other institutions make map layers available through ArcGIS Server map services. The United States Geological Survey’s National Map Program, for example, makes services available for adding basemap, topography, weather and other layers. These map layers can often be a useful addition to your own online maps and a relatively […]

Continue reading →

Spatial data on a diet: tips for file size reduction using TopoJSON

Posted on April 22, 2014 by zev@zevross.com

For a current project we needed to render, on the web, a geographic file with approximately six thousand features. The shapefile itself is approximately 11 megabytes, far too big to be handled speedily in a web application. We were planning to render the features using GeoServer and map tiles but decided to investigate slimming down […]

Continue reading →

Putting TopoJSON on a Google Map

Posted on April 14, 2014 by zev@zevross.com

For a recent project we needed to significantly reduce the file sizes of some of the GeoJSON files were were using so we turned to Mike Bostock’s amazing TopoJSON APIs. As described on the wiki page TopoJSON is an extension of GeoJSON that significantly reduces file sizes by reducing redundancy. For example, the border between […]

Continue reading →

Using R to quickly create an interactive online map using the `leafletR` package

Posted on April 11, 2014 by zev@zevross.com · 3 Comments

In a recent post (which you can find here) we identified the first publish date for all spatial packages listed in the Analysis of Spatial Data Task View on the R website. The most recent of these, published in March 2014, is the leafletR package by Christian Graul. We were surprised and impressed that, if […]

Continue reading →

Extracting tabular data from a PDF: An example using Python and regular expressions

Posted on April 9, 2014 by zev@zevross.com · 3 Comments

It is not uncommon for us to need to extract text from a PDF. For small PDFs with minimal data or text it's fairly straightforward to extract the data manually by using 'save as' or simply copying and pasting the data you need. For a recent project, however, we were asked to extract detailed address […]

Continue reading →

Interactive visualization: from R to D3 using `rCharts`

Posted on April 3, 2014 by zev@zevross.com · 1 Comment

Data Driven Documents, or D3 for short, is an incredible JavaScript library for creating interactive data visualization on the web. Earlier this year, for example, we illustrated the power of D3 by interactively linking maps and charts in this visualization. D3, however, can be challenging to work with, especially if you don't have experience with […]

Continue reading →

Google Maps API Adds GeoJSON Support: Here is an example

Posted on April 1, 2014 by zev@zevross.com · Leave a comment

In mid-March, 2014 Google announced that it added support for GeoJSON to the Google Maps API (v3). Although other mapping APIs, namely Leaflet, have supported GeoJSON data for quite some time, easy access to GeoJSON in Google Maps will simplify the coding lives of many developers who often use the Google Maps API. In the […]

Continue reading →

Technical Tidbits From Spatial Analysis & Data Science

Month: April 2014

For large tables in R `dplyr's` function `inner_join()` is much faster than `merge()`

R, Python, PostgreSQL (and more): A data science workflow example

Adding ArcGIS map services to your map using Leaflet

Spatial data on a diet: tips for file size reduction using TopoJSON

Putting TopoJSON on a Google Map

Using R to quickly create an interactive online map using the `leafletR` package

Extracting tabular data from a PDF: An example using Python and regular expressions

Interactive visualization: from R to D3 using `rCharts`

Google Maps API Adds GeoJSON Support: Here is an example