Welcome Sign in
MSDN Magazine
 Printer Friendly Version      Send     
Click to Rate and Give Feedback
Related Articles

Here the author introduces SQL Server Data Services, which exposes its functionality over standard Web service interfaces.

David Robinson

MSDN Magazine July 2008

...

Read more!

Here we explain how the new hierarchyID data type in SQL Server 2008 helps solve some of the problems in modeling and querying hierarchical information.

Kent Tegels

MSDN Magazine September 2008

...

Read more!

Here Brian Randell presents everything you need to know to get started with Visual Studio 2005 Team Edition for Database Professionals.

Brian A. Randell

MSDN Magazine February 2007

...

Read more!

The Entity Framework is a hot topic, but how do you know when to use EntityClient, Object Services, Entity SQL, or LINQ? Find out here.

John Papa

MSDN Magazine May 2008

...

Read more!

Building a data access layer using LINQ to SQL and the ADO.NET Entity Framework allows you to decouple your application from the persistence technology you're using.

Anthony Sneed

MSDN Magazine December 2008

...

Read more!

Popular Articles

Learn how to automate custom SharePoint application deployments, use the SharePoint API, and avoid the hassle of custom site definitions.

E. Wilansky, P. Olszewski, and R. Sneddon

MSDN Magazine May 2008

...

Read more!

Here we present techniques for programmatic and declarative data binding and display with Windows Presentation Foundation.

Josh Smith

MSDN Magazine July 2008

...

Read more!

Now you can perform efficient, sophisticated text analysis using regular expressions in SQL Server 2005.

David Banister

MSDN Magazine February 2007

...

Read more!

Kenny Kerr sings the praises of the new Visual C++ 2008 Feature Pack, which brings modern conveniences to Visual C++.

Kenny Kerr

MSDN Magazine May 2008

...

Read more!

Here are some design patterns that allow you to achieve higher cohesion and looser coupling for more flexible, reusable applications.

Jeremy Miller

MSDN Magazine October 2008

...

Read more!

Our Blog

Because Windows Workflow Foundation (WF) is based on a runtime that manages the execution of workflows and activities, testing must, in almost all cases, involve the use of the runtime – and this can introduce some interesting challenges.

In the November 2008 issue of MSDN Magazine, Matt Milner presents some techniques for unit testing ...

Read more!

Silverlight and SharePoint provide a simple, yet powerful, infrastructure for building intranet and extranet applications with sophisticated user interface designs and interactions.

In the November 2008 issue of MSDN Magazine, Steve Fox and Paul Stubbs demonstrate how to build a SharePoint Web Part as a wrapper for a Silverlight application.

...

Read more!

With the releases of LINQ to SQL and the ADO.NET Entity Framework, developers now have two products from Microsoft designed to tie together relational data and object-oriented programming.

In the December 2008 issue of MSDN Magazine, Anthony Sneed provides a roadmap to these technologies and demonstrates how you can create ...

Read more!

So many factors can affect the performance of a Web page—the distance between server and client, the size of the elements on the page, how the browser loads these elements, available bandwidth. Finding those bottlenecks and identifying the culprits is no easy task.

In the November 2008 issue of MSDN Magazine, Jim Pierson introduces ...

Read more!

Windows Presentation Foundation (WPF) adds functionality to the Microsoft .NET Framework so that you actually can reliably keep bound controls synchronized with their data sources.

In the December 2008 issue of MSDN Magazine, Ken Getz demonstrates how to use the ObservableCollection class provided by WPF to keep bound controls in ...

Read more!
{End Bracket}
The World-Wide Telescope
Jim Gray and Alexander Szalay


Someday all scientific literature and data will be online and accessible to everyone everywhere. The astronomy community has made unusually good progress toward this vision of online science in addition to addressing the associated challenges of data publication.
Much of astronomy involves comparing data from many instruments (from different parts of the electromagnetic spectrum) taken at different times. The Crab Nebula provides a good example of the temporal and multispectral nature of astronomy. The Crab Star supernova was first observed on July 4, 1054. Now we see it as the Crab Nebula—a gas cloud expanding at relativistic speeds. The cloud's core appears to be a black hole. Matter falling into this hole emits two energy beams that illuminate the cloud. Looking at this system in the X-ray spectral band shows the beams. Looking at the system in the optical and infrared spectral bands shows the gas escaping from the black hole. Each provides complementary information that together presents a fairly complete picture.
Astronomy obeys Moore's law: it is producing about two times more data each year. Current instruments typically produce nearly a terabyte per night. Managing huge data archives and processing complex data are now among the major astronomy challenges.
The Sloan Digital Sky Survey (SDSS) is a good example. It is a 5-band optical survey of the northern sky, observing about 400 million sources as images and 1 million with spectra. These spectra allow detailed studies of large star and galaxy populations. The traditional way of accessing this data is to place it in files and let users FTP the files they want to their local systems for analysis.
We built an online catalog of the SDSS data as a Web-accessible database, along with visual tools to analyze the data (SkyServer.sdss.org). The result is a SQL Server™ database with approximately 14 billion rows. It gives full GUI and SQL access to the SDSS data. Now everyone can use one of the world's best telescopes. The site has been a big success—about 10 percent of the visitors are students using online courses, but the main users are astronomers analyzing the available data.
SkyServer must support many browsers running on many platforms, so we took a thin-client approach in which most processing is done on a server that produces standard HTML. Much of the server-side logic is implemented in T-SQL stored procedures. The Web services rendering images are coded in C#. The database schema is self-documenting, and the design allows users to plug their catalog and image data into a spatial search framework and Web service. A C# variant of the spatial data search is part of the SQL Server 2005 samples. You can download a personal copy of all the SkyServer code (Web site and database) along with a version of the database from SkyServer.org.
SkyServer has quite a few Web services to give users programmatic access to the data and analysis tools, a classic service-oriented architecture. But some of the astronomy queries run for hours, so we set up a system to let users submit long-running jobs (the Catalog Archive Serve Jobs System, or CASJOBS). We also allowed users to create personal databases (MyDB) near the server. MyDB stores intermediate results and uploaded user data, allowing users to do multistep analyses on huge datasets. The article "Batch is back: CasJobs, serving multi-TB data on the Web,", describes this system.
The International Virtual Observatory Alliance is a grassroots group of astronomers who want to federate world-wide astronomy data, cross-index it with the literature, and provide analysis tools to the community. The SDSS data is now part of this federation. A user can point to a SkyServer object and find all the literature on that object, as well as all other public data sets about that object. This is all architected using Web services defined by the IVOA. The IVOA is also defining a standard schema and ontology for astronomy data (www.ivoa.net/Documents).
We are now much closer to reaching the goal of the World-Wide Telescope federating all the world's astronomy data and literature. OpenSkyQuery.Net offers a prototype federating 29 archives. The astronomy community is well on the way to building the next generation of this design.

Jim Gray works at Microsoft Research and focuses on eScience scaleable systems and databases. He has been active in building online databases like terraService.Net and SkyServer.sdss.org. See research.microsoft.com/~gray.

Alexander Szalay is a Professor at the Johns Hopkins University, both in the Department of Physics and Astronomy, and the Department of Computer Science. He is the Project Director of the NSF-funded National Virtual Observatory. See www.sdss.jhu.edu/~szalay.
Page view tracker