23 votes

Rank25

Idea#95

This idea is active.
App/Services "Killer Ideas" »

Building DOE Infrastructure for Web Services and Linked Data

My idea is to have the DOE work in a public/private partnership with a group of expert information system architects who can help the DOE develop open Web services and linked data approaches to achieving th American Energy Data Challenge vision. Ideally, the DOE's diverse databases and data streams will become a seamless part of an even larger pool of energy data that can be tapped for everything from simple charts to high powered data analytics services in the cloud. Stakeholders will be able to easily publish, discover, assess, filter, access, and use energy-related data regardless of the data's source.

Most energy-related data records have a spatial component, as do most end uses for the data. Creating end-use information products from DOE data or similar non-DOE data will usually involve filtering energy data for a specific geospatial point or region. I propose that the DOE undertake this energy data access project in the not-for-profit Open Geospatial Consortium (OGC). In the OGC, 10 US agencies participate with 465 other public and private sector organizations in a consensus process to develop and advance open geospatial encoding and interface standards. The OGC has an Energy & Utilities Domain Working Group. Also, the OGC leads the multi-year, multi-nation GEOSS Architecture Implementation Pilot (AIP). Some of the OGC Engineering Reports that come out of the AIP (http://www.ogcnetwork.net/AIP2ERs) are a product of the Renewable Energy Societal Benefit Area activity.

In the OGC, the DOE and other stakeholders could share the costs of an energy data testbed. Testbed sponsors would create a set of real-world use cases and scenarios that would guide technology providers in developing example solutions. After a fast-paced prototyping and testing effort, they would demonstrate how these work through open encodings, interfaces and best practices.

Downloading files is an outdated practice. 21st century Web services and linked data offer much more capability. Perhaps the DOE would want to focus on Web service interfaces that make it easy for any geospatial software to access data records by queries to diverse federated data collections stored as CSV files. The DOE could make Green Button data (depersonalized, perhaps through averaging values for the nearest 100 residential ratepayers) available as a spatially-queryable web service. Perhaps the DOE would ask for standards, best practices and reference implementations that would use linked data to support cities in creating integrated community energy systems.

If I were awarded a prize for this idea, as OGC's Senior Staff Writer I would use the money to help this work advance, according to terms to be determined by agreement between a DOE representative and OGC's president and CEO.

Submitted by lancemckee 8 months ago

Vote Activity Show

(latest 20 votes)

Comments (4)

  1. I agree with Lance that GeoSpatial Web Services and Linked Data are the appropriate technologies to use to reduce the effort and expense required for more widespread use and analysis of and benefit from energy data. They go far beyond file downloads, which web services can also provide, with APIs for targeted search and on-line application support. Explore IOOS from NOAA http://www.ioos.noaa.gov/ and flood maps from FEMA http://www.floodsmart.gov/floodsmart/ to see some examples of what's possible using OGC GeoSpatial Web Services. For example, some FEMA web apps make real time uses of services provided by The National Map http://nationalmap.gov/ from USGS. It's long past time for web analytics to be able to access energy data in a similar fashion.

    7 months ago
  2. Yes, I agree. An efficient energy grid will require interoperable access to and monitoring of a large number of different devices (e.g. sensor, actuators), computational model, and processes. Current access to these assets is highly diverse ("stovepiped") and standard web services and encodings from OGC would greatly improve our ability to monitor power consumption, detect failures or "leaks", monitor the environment (e.g. weather, river levels, etc.), and provide input and output to and from models and simulations that will be needed to predict needs over the next days to months, and to appropriately respond to these needs. It is just too challenging to gather, process, and respond to the valuable information that is available from all of these assets, but is difficult to integrate into tools and processes. In OGC, many standards (e.g. KML, Web Feature Services, SensorML, etc.) have been developed that have definite relevance to our ability to provide decoupled but integrated access to components of our future energy grid.

    7 months ago
  3. I agree with Lance's recommendation that the DoE migrate the "way" in which energy information is shared. Once these best practices in information architecture are in place, the fundamental foundation necessary for sustained, scale-able solutions will be in place.

    7 months ago
  4. Web services and linked data interoperability is the basis for systems' integration as needed and is flexible to evolve with changing requirements.

    7 months ago