Sponsors

  • Microsoft
  • Nebula
  • Google
  • SugarCRM
  • Facebook
  • HP
  • Intel
  • Rackspace Hosting
  • WSO2
  • Alfresco
  • BlackBerry
  • CUBRID
  • Dell
  • eBay
  • Heroku
  • InfiniteGraph
  • JBoss
  • LeaseWeb
  • Liferay
  • Media Temple, Inc.
  • OpenShift
  • Oracle
  • Percona
  • Puppet Labs
  • Qualcomm Innovation Center, Inc.
  • Rentrak
  • Silicon Mechanics
  • SoftLayer Technologies, Inc.
  • SourceGear
  • Urban Airship
  • Vertica
  • VMware
  • (mt) Media Temple, Inc.

Sponsorship Opportunities

For information on exhibition and sponsorship opportunities at the convention, contact Sharon Cordesse at scordesse@oreilly.com

Download the OSCON Sponsor/Exhibitor Prospectus

Contact Us

View a complete list of OSCON contacts

RESTful Web Services for Scientific Computing

Joshua Boverhof (Lawrence Berkeley National Lab), Shreyas Cholia (NERSC)
Python
Location: D133
Average rating: **...
(2.33, 3 ratings)

The National Energy Research Scientific Computing Center (NERSC) houses some of the worlds most powerful supercomputers. NERSC awards HPC cycles via an application process, among the scientific applications consuming cycles are climate change and quantum mechanics simulations. At NERSC several scientific portal efforts are underway in order to open up HPC resources to a wider user community.

The typical NERSC user will remotely log into an HPC system, and in a shell environment using command-line tools create an application’s input files, and a PBS script which will be submitted to the batch scheduler. Depending on the queue, the scheduler may take hours to move the job to run status. After the job has completed, which may take hours or weeks, the various output files generated by the application may need to be moved and analyzed. The record of the job run is the working directory where all the input/output files are kept. This is a typical workflow for an HPC user at NERSC.

There are various limitations of a command-line centric world. First many users, especially those in emerging or growing HPC Scientific Application communities, don’t like and don’t want to learn command-line. Their comfort zone is the World Wide Web, learning to use a new web site is preferable than reading and rereading documentation on how to submit and monitor HPC jobs at NERSC. A portal environment provides a single point of access, which is extremely advantageous for recording a history, canning workflows, and combining the power of underlying resources into a unified interface. Secondly, command-line tools are computer-centric while user’s want application-centric tools. The NERSC Web Toolkit (NEWT) provides RESTFul HTTP interfaces to HPC resources, scientific application users and portal developers can utilize these services to create new user facing web applications.

In this talk I will present the NEWT API, implemented as a python Django project, and discuss NERSC portals that are taking advantage of it.

Photo of Joshua Boverhof

Joshua Boverhof

Lawrence Berkeley National Lab

Joshua is an engineer at LBNL, his expertise includes web and cloud technologies. He is currently developing RESTful web services for the NERSC supercomputing center, and works on “globus.org” a hosted managed file transfer service.

Shreyas Cholia

NERSC

Shreyas Cholia works on science gateway, web and grid technologies for NERSC, with the goal of making high-performance and distributed computing more transparent and accessible. He has also been involved in various grid and data-driven science efforts at NERSC since 2002. Prior to his appointment at NERSC, Shreyas was a developer and consultant at IBM. He has a bachelor’s degree from Rice University, where he double majored in Computer sciences and Cognitive Sciences.