My idea is to have the DOE work in a public/private partnership with a group of expert information system architects who can help the DOE develop open Web services and linked data approaches to achieving th American Energy Data Challenge vision. Ideally, the DOE's diverse databases and data streams will become a seamless part of an even larger pool of energy data that can be tapped for everything from simple charts to high powered data analytics services in the cloud. Stakeholders will be able to easily publish, discover, assess, filter, access, and use energy-related data regardless of the data's source.
Most energy-related data records have a spatial component, as do most end uses for the data. Creating end-use information products from DOE data or similar non-DOE data will usually involve filtering energy data for a specific geospatial point or region. I propose that the DOE undertake this energy data access project in the not-for-profit Open Geospatial Consortium (OGC). In the OGC, 10 US agencies participate with 465 other public and private sector organizations in a consensus process to develop and advance open geospatial encoding and interface standards. The OGC has an Energy & Utilities Domain Working Group. Also, the OGC leads the multi-year, multi-nation GEOSS Architecture Implementation Pilot (AIP). Some of the OGC Engineering Reports that come out of the AIP (http://www.ogcnetwork.net/AIP2ERs) are a product of the Renewable Energy Societal Benefit Area activity.
In the OGC, the DOE and other stakeholders could share the costs of an energy data testbed. Testbed sponsors would create a set of real-world use cases and scenarios that would guide technology providers in developing example solutions. After a fast-paced prototyping and testing effort, they would demonstrate how these work through open encodings, interfaces and best practices.
Downloading files is an outdated practice. 21st century Web services and linked data offer much more capability. Perhaps the DOE would want to focus on Web service interfaces that make it easy for any geospatial software to access data records by queries to diverse federated data collections stored as CSV files. The DOE could make Green Button data (depersonalized, perhaps through averaging values for the nearest 100 residential ratepayers) available as a spatially-queryable web service. Perhaps the DOE would ask for standards, best practices and reference implementations that would use linked data to support cities in creating integrated community energy systems.
If I were awarded a prize for this idea, as OGC's Senior Staff Writer I would use the money to help this work advance, according to terms to be determined by agreement between a DOE representative and OGC's president and CEO.