There are a good number of datasets on the open data portal that are made available as zipped shapefiles. The shapefile format was developed by Esri to support interoperability among various Geographic Information Systems (GIS) software. You can learn more about shapefiles on Wikipedia.
Shapefiles can be a bit confusing to individuals that have not worked with geographic data before. For example, a shapefile is actually not a single file, but a bundle of files representing different aspects of geographic data.
There are a number of ways to work with shapefiles that range based on your needs. Below are different ways you can open and interact with this kind of data. The following are provided for your information only. We do not offer an endorsement of any of the approaches or software and advise you to pick the one that works best for you. This is a rapidly changing area, so please remember to do a quick search.
Opening and viewing the map data (points, lines and polygons) contained in the shapefile
- Esri provides a subscription to their software for personal use, the Esri line of products will open, manipulate and produce shapefiles and other geographic formats (Windows)
- QGIS is an open source and free piece of software that will also allow you to open, manipulate and produce shapefiles and other geographic formats (Windows, Mac, Linux)
- The same subscription above for Esri gets you access to a named user account on ArcGIS.com which provides web-based mapping as well
- CartoDB provides a freemium subscription service that lets you upload shapefiles and view the geometries on a map as well as the underlying attributes
- MapBox Studio provides a method to load shapefiles as tilesets where you can further manipulate and explore the data.
- Mapshaper is an open source tool for loading and editing geographic data in a couple of formats including shapefile. You can try it online or if you're adventurous, download and run it locally.
Dealing with projections
A map projection is a "systematic transformation of the latitudes and longitudes of locations on the surface of a sphere or an ellipsoid into locations on a plane" (from Wikipedia). It's what allows one to take geographic data and flatten it out. There are different projections that distort data in different ways. Generally, most users will be well served by WGS84/Web Mercator projection for things like web mapping. But your needs may vary based on how you need to use the data.
Projections can take a little bit to understand, but most software and tools will handle changing projections for you if you know the input and the desired output. To understand the input projection you can refer to the *.prj file that comes bundled in the zipped download. You can even open this file and see the projection in any text editor. You'll see the file will start with PROJCS["<name_of_projected_coordinate_system>". Many of the datasets in zipped shapefiles are provided in NAD 1983 StatePlane California III FIPS 0403 Feet.
If you have any questions or do not see the *.prj file, please reach out to us via the help desk and we'll work on getting the issue resolved.
Converting shapefiles to geojson
If you know what geojson is, then these resources may be useful to you:
- Ben Balter of GitHub provides a quick tutorial on how to transform a shapefile to geojson. He also details how to load that to github where you can quickly visualize, explore and use the data in your projects
- In the above, Ben references how to use the command line to convert shapefiles to geojson. If you don't want to wrestle with that, try an online version of ogr2ogr (the commandline tool) called Ogre
- You can also use QGIS referenced above to open a shapefile and save as geojson
I just want the tables, not the geometry
The tabular data is contained in the DBF which is part of a shapefile. This is no longer a well-supported tabular format and there are fewer and fewer readily available tools to open just this file. That said, most of the options above will pull out the attributes related to each point, line or polygon, which you should be able to extract to a table.
Can't DataSF just give me a more useful format?
Well, glad you asked! We're providing this article to help our data users in the meantime, but we are in the midst of moving our geographic data to more accessible formats. You'll still be able to get a shapefile if that's your format of choice, but there'll be other formats including geojson. In fact, there are already datasets available in this way on the open data portal.
We are transferring our datasets to more accessible formats on a rolling basis. Those that are updated more frequently may take a little longer as we develop automated jobs to keep the data fresh on the portal. Stay tuned!