Windows High Performance Computing (HPC) and Programming Resources

Since the last year, I was researching about distributed and grid computing. I found many useful resources and information (links at end of this post). One of the topics I found is Microsoft implementation of High Performance Computing (HPC). This post is a list of resources I think are relevant to the

First, the page of Windows HPC Server 2008:

http://www.microsoft.com/hpc

Image may be NSFW.
Clik here to view. The first video to watch is the last week PDC 2008 Session:

HPC Session at last PDC
http://channel9.msdn.com/pdc2008/ES13/

It’s an excellent presentation, covering the new Windows HPC Server 2008, nodes, tasks and jobs, management tools, programming options, MPI and MPI.NET programming, computer go on HPC (beautiful idea), all the presentation deserves a dedicated post.

I like a short but interesting video, showing the management console, at:

http://channel9.msdn.com/shows/The+HPC+Show/Five-Minute-Intro-to-the-HPC-Server-2008-Management-Console/

THE blog to read is Windows HPC survival guide

A post as example: No scientist left behind with CRAY Supercomputer running Windows HPC Server 2008

They collected a set of resources at HPC Resource Kit

All the videos related to HPC at:

HPC | Tags | Channel 9

(interesting topics: WCF and HPC programming, HPC Basic Profile: open web services you can invoke from Java and other languages)

There is a community site dedicated to Windows HPC:

http://www.windowshpc.net/

with files, resources, source code and examples. 

Software to use

To start writing software for HPC, install Microsoft HPC Pack (Windows). I downloaded it from:

HPC Pack 2008 SDK download

and then, install MPI.NET Software

(I installed MPI.NET SDK.msi but I expanded MPI.NET-1.0.0.zip: it has better examples, with VS solutions)

You don’t need the HPC server to run these examples.

An excellent tutorial, implementing a fractal application using HPC 2008 at:

Learning Parallel Programming — from shared-memory multi-threading to distributed-memory multi-processing

Additional Links

If you want to explore the HPC programming possibilites, these are the topics to research:

HPC

http://www.hpccommunity.org/ HPC Community
http://www.hpcwire.com/ High Productivity Computing
http://www.ddj.com/hpc-high-performance-computing/
YouTube – An Overview of High Performance Computing and Challenges for the Future
http://en.wikipedia.org/wiki/High-performance_computing

MPI

MPI (Message Passing Interface) is supported by Windows HPC. There is a Microsoft implementation:

Microsoft MPI (Windows)

that can be invoked from C++.

There is a .NET implementation over Microsoft MPI:

MPI.NET: High-Performance C# Library for Message Passing

It has source code and examples.

(An old .NET wrapper at Codeplex project:  MPI .Net – Home)

I posted about other .NET implementation:

MPI Message Passing Interface in .NET

More about MPI

MPI 2.0 Report
MPI Tutorials
Microsoft Messaging Passing Interface – Wikipedia, the free encyclopedia
Pure Mpi.NET

Parallel Programming

Introduction to Parallel Computing very complete resource, (thanks to jgarcia)
Microsoft Innovation Day – November 5, 2006 they are presenting something related to DryadLINQ
Multithreading and Concurrency in .NET a very complete list of technologies available in .NET
http://www.microsoft.com/ccrdss Now, CCR/DSS as a separated package (formerly in Microsoft Robotics)
Adobe Press – 9780321603944 – Software Pipelines: The Key to Capitalizing on the Multi-core Revolution
Burton Smith: On General Purpose Super Computing and the History and Future of Parallelism | Going Deep | Channel 9
Welcome to Hadoop!
Dryad – Home A Microsoft research project
YouTube – Dryad: A general-purpose distributed execution platform Presentation at Google Talks
Concurrency: What Every Dev Must Know About Multithreaded Apps
Overview of concurrency in .NET Framework 3.5 | Igor Ostrovsky Blogging
Parallel Programming with .NET
Parallel Computing Developer Center from Microsoft
Parallel Virtual Machine – Wikipedia, the free encyclopedia
http://msdn.microsoft.com/msdnmag/issues/07/10/PLINQ/default.aspx Parallel LINQ

Map Reduce

Writing An Hadoop MapReduce Program In Python
Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks
Google Research Publication: MapReduce

Delicious

My delicious links about HPC, MPI, Parallel programming, Grid Computing, Map Reduce algorithms, CCR/DSS:

http://delicious.com/ajlopez/hpc
http://delicious.com/ajlopez/mpi
http://delicious.com/ajlopez/parallel
http://delicious.com/ajlopez/gridcomputing
http://delicious.com/ajlopez/mapreduce
http://delicious.com/ajlopez/ccr
http://delicious.com/ajlopez/dss

Computer Go is a fascinating topic:

http://delicious.com/ajlopez/computergo

Angel “Java” Lopez
http://www.ajlopez.com/en
http://twitter.com/ajlopez

Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view.