News

July 20, 2008: Powered by UDT?We are setting up a web page to list projects that uses UDT for data and message transfer. Please drop me an email (gu#lac.uic.edu) about how you use UDT in your project if you are interested. (Note: Whether you send me the information or not does NOT affect your right in using UDT.) This will help us promote the UDT protocol and further improve it based on users' feedbacks. Thanks for your input.

July 20, 2008: UDT supports communication in large scale cloud computingUDT is used in cloud computing software Sector/Sphere for both message passing and data transfer. UDT provides better scalability, performance, and security than Linux TCP in this system. We have installed Sector/Sphere in a 120-node system deployed across the United States. In this system, more than 120*120 concurrent UDT connections are used for data exchanging during a data intensive distributed application.

June 28, 2008: Announcing Sector and SphereWe are glad to announce a new open source software suite, Sector/Sphere, a high performance distributed storage and processing system, available at http://sector.sourceforge.net. Sector supports distributed data storage/file system and can also be used a content distribution network. Sphere supports simplified programming of data parallel applications, similar to MapReduce.

Sector/Sphere has similar functionality as GFS/MapReduce and Hadoop but differs significantly in design and implementation details. Considering using Sector rather than Hadoop if you are more familiar with C++ than Java or you need to deploy the system over wide area (e.g., for content distribution or data sharing). Meanwhile, Sector is simpler (hence better potential of reliability) and faster (2 - 4 times faster for Terasort).

Sector features all-UDP communication and UDT is used for data transfer.

November 9, 2007: Visit us at the upcoming Supercomputing Conference!National Center for Data Mining will be at the annual Supercomputing Conference to be held in Reno, NV during November 11 - 17. We will demonstrate UDT version 4 for high speed data transfer over International network testbeds, together with other high performance distributed applications. UDT will also be used in our entry to this year's Bandwidth Challenge. You are welcome to visit us at exhibition booth #2623.

October 26, 2007: UDT version 4.0 Release UDTv4 is the latest version of the new versatile high speed UDP-based data transfer protocol. One of the major improvement from UDTv3 is that UDTv4 can bind multiple UDT sockets on the same UDP port. UDTv4 also greatly improved the configurable congestion control (CCC) module as its native control algorithm is also written based on CCC. Finally, in order to reach a wider audience, we have released UDTv4 under BSD license.

August 21, 2007: UDT Commercial License to be Available We are planning to introduce a commercial license for UDT via the University of Illinois at Chicago. UDT will be dual-licensed. We will continue to release UDT under LGPL. However, a commercial license will be available for users who cannot or do not prefer to use LGPL. Details will be available soon. Meanwhile, please contact Yunhong Gu (gu @ lac.uic.edu) if you have any comments, requests, or questions.

[Note] this commecial license plan has been put on hold for indefinite length of time.

June 4, 2007: USA-Russia Lightpath Enables Fast Data Transfer of Terabyte-sized Scientific Datasets UDT helped to reach 711Mb/s (peak 844Mb/s) disk-disk data transfer between US and Russia. Scientists from the National Center for Data Mining (NCDM) at the University of Illinois at Chicago and the Geophysical Center at the Space Research Institute, Russian Academy of Sciences, Moscow, demonstrated a new method for distributing extremely large volumes of scientific information across the world. They successfully moved 1.4 TeraBytes (TB) of data in about 4.5 hours over a 1 Gbps lightpath between Chicago and Moscow as part of the Teraflow Network initiative. This event, which represents the highest performance information transfer ever recorded between these two countries, was made possible by a unique international organizational partnership. More...

November 17, 2006: UDT Won SC06 Bandwidth Challenge Award UDT was used in the bandwidth challenge winning entry "Transferring Sloan Digital Sky Survey Data using SECTOR" during the annual Supercomputing conference in Tampa, FL. UDT enabled a disk to disk data transfer at 9.18Gb/s peak throughput between Chicago and Tampa.

May 23, 2006: UDT v3.0 Released Version 3.0 is released. The code is reorganized and many are rewritten. Composable UDT framework is included in this release.

March 24, 2006: Datagram Mode Supported by UDT UDT now supports partial reliable messaging. In the new SOCK_DGRAM mode, the message boundaries are preserved and for each message, applications can specify a time-to-live value and a boolean flag to tell if the message should be delivered in order. This new scheme is included in version 3.0, which has a beta release now available.

The reliable data streaming mode is still supported, of course. It must be explicitly enabled with SOCK_STREAM mode now. A reliable version 2.4 was just released.

December 12, 2005: Rendezvous Connection Setup Supported The most recent effort of the UDT development is to support firewall punching, especially those NAT firewalls.

September 28, 2005: UDT Demo at iGrid 2005 Scientists from the National Center for Data Mining (NCDM) at the University of Illinois at Chicago established a new record for transferring nearly a terabyte of astronomy data, disk-to-disk, across the Pacific, using their UDT protocol. NCDM and its partners achieved this milestone at iGrid 2005, in San Diego, CA. More...

October 21, 2004: UDT 2.0 ReleasedVersion 2.0 is the first production version and UDT starts its own project on SourceForge.

October 17, 2003: UDT Set Data Transfer Milestone A new milestone was reached in trans-Atlantic data transmission by researchers at the University of Illinois at Chicago (UIC) who demonstrated the practicality of transferring even very large data sets over high-speed production networks.

UIC's National Center for Data Mining (NCDM) and Laboratory for Advanced Computing flashed a set of astronomical data across the Atlantic at 6.8 gigabits per second --- 6800 times faster than the 1 megabit per second effective speed that connects most companies to the internet.

In the test, 1.4 terabytes of astronomical data was transmitted from Chicago to Amsterdam in 30 minutes using UDT, a new protocol developed by the NCDM at the University of Illinois at Chicago. In comparison, moving the same amount of data using the TCP Protocol, which is the standard used on the internet today for data transfers, would take 25 days.

See more media coverage from HPC wire, Science Daily, Chicago Sun Times, etc.

October 30, 2002: Data transfer demo sets speed markResearchers from Northwestern University and the University of Illinois at Chicago have set a new speed record for transmitting data: 2.8 gigabits, or billion bits, per second. The researchers set the record while transmitting information between Amsterdam and Chicago at the iGRID 2002 conference in Amsterdam on September 24.

Home | Contact Us | © 2007 National Center for Data Mining. All rights reserved.