Dec. 29, 2011: UDT v4.10 release!It has been a relatively quiet year on UDT but we have managed to introduce some non-trivial features in version 4.9 and 4.10. We have greatly imroved the epoll API and also added support to non-blocking connect API. In 2012, we will try harder to introduce UDT to more use scenarios.
Dec. 29, 2010: UDT v4.8 release! UDT version 4.8 introduces new epoll API, which significantly improves the scalabity on concurrent connections. Our experiments show that now UDT can easily support 100K+ connections via a single UDP port on a commodity server (buffer size needs to be reduced accordingly). Thanks every UDT user for your continuing support.
June 14 , 2010: A High Performance Data Distribution and Sharing Solution with Sector We invite you to try Sector as your data distribution and sharing platform. Unlike FTP and most commercial applications, Sector can provide a uniform namespace over multiple servers, even 1000s. Meanwhile, users can mount the client side as a local file system directory so that it is completely transparent. All data transfer between servers and clients is on top of UDT. You can follow this blog to set up the system.
June 09, 2010: Monitoring UDT with Argus Argus, the network Audit Record Generation and Utilization System, now supports throughput, goodput, loss, jitter and window advertisement reporting for UDT. Argus provides packet level monitoring and audit and it is a great addition to the UDT performance monitoring function. The Argus system is developed by QoSient LLC and is available at here.
Jan. 06, 2010: UDT Commercial Service Now Available from VeryCloud LLC Users can now obtain various level of dedicated services for UDT from VeryCloud LLC. The services include but not limited to programming tutorial/training, code explanation, debugging/diagnosis, code customization, and new algorithm/system design and development. Founded by the architect and developers of UDT and Sector/Sphere, VeryCloud provides products and services on high performance data transfer, storage, and processing.
Nov. 19, 2009: UDT supported SC09 Bandwidth Challenge Winning EntryDuring the annual Supercomputing conference (SC09), the joint team of National Center for Data Mining (NCDM), Naval Research Laboratory (NRL), and International Center for Advanced Internet Research (iCAIR) won the 10th Bandwidth Challenge competition. UDT was used to support high speed data transfer in several wide area applications demonstrated by the joint team. NRL implemented the UDT protocol on FPGA to support long distance Infiniband, while NCDM and iCAIR demonstrated a light weight UDT variant, UDX, for ultra-high speed networks.
Sep. 19, 2009: Meet Us at SC09: Nov 14 - 20 in Portland, OR The NCDM team will be at the annual supercomputing conference (SC09) in Portland, OR, during Nov 14 - 20. We would like to meet you and discuss UDT, Sector/Sphere, and more on high performance computing and data analytics. We are located at booth 1309 on the exhibition floor.
July 29, 2009: UDT Consulting Now AvailableWe now offer UDT consulting services, including but not limited to: 1) system diagnosis and tuning for high speed data transfer; 2) UDT customization to fit your specific application requirements; and 3) UDT development training. Please email Yunhong Gu <yunhong.gu at gmail dot com> with a brief description of your system. Please note that this service is NOT associated with UIC or NCDM.
July 10, 2009: UDT Version 4.5a Release We have released version 4.5a, which includes several critical bug fixes since the last version. An updated congestion control algorithm is introduced to help UDT work better in low bandwidth environments, while still achieving high performance in high speed networks. Version 4.5a also introduces the new selectEx API, which can handle large number of UDT sockets efficiently. Thanks to everyone for helping us improve the software!
May 22, 2009: UDT and Cloud Computing UDT was featured in a press release by National Science Foundation (NSF) titled "Data Travels Six Times Faster in the Clouds": The National Center for Data Mining (NCDM) at the University of Illinois at Chicago established a cloud computing system (Sector) that can quickly compile data from widely geographically distributed data centers across high performance networks. Sector employs an alternative network protocol called UDT designed to swiftly and smoothly transfer data. Full story...
May 21, 2009: Black Hole Simulation Wins SCALE 2009 ChallengeLSU-led Black Hole demonstration won first prize at the SCALE 2009 challenge at CCGrid09, a premier conference for cluster and Grid computing. UDT was used to support data transfer in this project.
January 22, 2009: UDT Version 4.4 Release We have released version 4.4, which includes several critical bug fixes since the last version. Please note that we introduced two new APIs, startup and cleanup, in this version. Our next version will address the efficiency issue for very large number of concurrent connections. Thanks to everyone for helping us improve the software.
November 24, 2008: The UDT team have won the bandwidth challenge (again!) during the Supercomputing conference During the supercomputing 2008 (SC08) conference, we demonstrated three cloud applications (Cistrack - a bioinformaics portal, CreditStone - processing credit card transaction records, and Terasort). These applications are supported by our cloud software stack Sector/Sphere and run on the Open Cloud Testbed. Complicated movement of terabytes of data is required in these applications, and UDT has supported this very well. More...
November 11, 2008: UDT Demo at SC08, Austin, TX From Nov. 16 to Nov. 20, our team will be at SC08, Austin, TX. We will demonstrate UDT and Sector/Sphere at the exhibitor floor, booth 321. We will also participate this year's bandwidth challenge using UDT and Sector/Sphere in various data intensive distributed applications. You are welcome to come to our booth. The UDT developer will answer your questions on site.
July 20, 2008: Powered by UDT?We are setting up a web page to list projects that uses UDT for data and message transfer. Please drop me an email (gu#lac.uic.edu) about how you use UDT in your project if you are interested. (Note: Whether you send me the information or not does NOT affect your right in using UDT.) This will help us promote the UDT protocol and further improve it based on users' feedbacks. Thanks for your input.
July 20, 2008: UDT supports communication in large scale cloud computingUDT is used in cloud computing software Sector/Sphere for both message passing and data transfer. UDT provides better scalability, performance, and security than Linux TCP in this system. We have installed Sector/Sphere in a 120-node system deployed across the United States. In this system, more than 120*120 concurrent UDT connections are used for data exchanging during a data intensive distributed application.
June 28, 2008: Announcing Sector and SphereWe are glad to announce a new open source software suite, Sector/Sphere, a high performance distributed storage and processing system, available at http://sector.sourceforge.net. Sector supports distributed data storage/file system and can also be used a content distribution network. Sphere supports simplified programming of data parallel applications, similar to MapReduce.
Sector/Sphere has similar functionality as GFS/MapReduce and Hadoop but differs significantly in design and implementation details. Considering using Sector rather than Hadoop if you are more familiar with C++ than Java or you need to deploy the system over wide area (e.g., for content distribution or data sharing). Meanwhile, Sector is simpler (hence better potential of reliability) and faster (2 - 4 times faster for Terasort).
Sector features all-UDP communication and UDT is used for data transfer.
November 9, 2007: Visit us at the upcoming Supercomputing Conference!National Center for Data Mining will be at the annual Supercomputing Conference to be held in Reno, NV during November 11 - 17. We will demonstrate UDT version 4 for high speed data transfer over International network testbeds, together with other high performance distributed applications. UDT will also be used in our entry to this year's Bandwidth Challenge. You are welcome to visit us at exhibition booth #2623.
October 26, 2007: UDT version 4.0 Release UDTv4 is the latest version of the new versatile high speed UDP-based data transfer protocol. One of the major improvement from UDTv3 is that UDTv4 can bind multiple UDT sockets on the same UDP port. UDTv4 also greatly improved the configurable congestion control (CCC) module as its native control algorithm is also written based on CCC. Finally, in order to reach a wider audience, we have released UDTv4 under BSD license.
August 21, 2007: UDT Commercial License to be Available We are planning to introduce a commercial license for UDT via the University of Illinois at Chicago. UDT will be dual-licensed. We will continue to release UDT under LGPL. However, a commercial license will be available for users who cannot or do not prefer to use LGPL. Details will be available soon. Meanwhile, please contact Yunhong Gu (gu @ lac.uic.edu) if you have any comments, requests, or questions.
[Note] this commecial license plan has been put on hold for indefinite length of time.
June 4, 2007: USA-Russia Lightpath Enables Fast Data Transfer of Terabyte-sized Scientific Datasets UDT helped to reach 711Mb/s (peak 844Mb/s) disk-disk data transfer between US and Russia. Scientists from the National Center for Data Mining (NCDM) at the University of Illinois at Chicago and the Geophysical Center at the Space Research Institute, Russian Academy of Sciences, Moscow, demonstrated a new method for distributing extremely large volumes of scientific information across the world. They successfully moved 1.4 TeraBytes (TB) of data in about 4.5 hours over a 1 Gbps lightpath between Chicago and Moscow as part of the Teraflow Network initiative. This event, which represents the highest performance information transfer ever recorded between these two countries, was made possible by a unique international organizational partnership. More...
November 17, 2006: UDT Won SC06 Bandwidth Challenge Award UDT was used in the bandwidth challenge winning entry "Transferring Sloan Digital Sky Survey Data using SECTOR" during the annual Supercomputing conference in Tampa, FL. UDT enabled a disk to disk data transfer at 9.18Gb/s peak throughput between Chicago and Tampa.
- SC06 Press Release: http://sc06.supercomputing.org/news/press_release.php?id=14
- Force10 Press Release: http://www.force10networks.com/news/pressreleases/2006/pr-2006-11-20.asp
- UIC Press Release: http://www1.cs.uic.edu/CSweb/public/news.php?audience=public&label=&ind=190
May 23, 2006: UDT v3.0 Released Version 3.0 is released. The code is reorganized and many are rewritten. Composable UDT framework is included in this release.
March 24, 2006: Datagram Mode Supported by UDT UDT now supports partial reliable messaging. In the new SOCK_DGRAM mode, the message boundaries are preserved and for each message, applications can specify a time-to-live value and a boolean flag to tell if the message should be delivered in order. This new scheme is included in version 3.0, which has a beta release now available.
The reliable data streaming mode is still supported, of course. It must be explicitly enabled with SOCK_STREAM mode now. A reliable version 2.4 was just released.
December 12, 2005: Rendezvous Connection Setup Supported The most recent effort of the UDT development is to support firewall punching, especially those NAT firewalls.
September 28, 2005: UDT Demo at iGrid 2005 Scientists from the National Center for Data Mining (NCDM) at the University of Illinois at Chicago established a new record for transferring nearly a terabyte of astronomy data, disk-to-disk, across the Pacific, using their UDT protocol. NCDM and its partners achieved this milestone at iGrid 2005, in San Diego, CA. More...
October 21, 2004: UDT 2.0 ReleasedVersion 2.0 is the first production version and UDT starts its own project on SourceForge.
October 17, 2003: UDT Set Data Transfer Milestone A new milestone was reached in trans-Atlantic data transmission by researchers at the University of Illinois at Chicago (UIC) who demonstrated the practicality of transferring even very large data sets over high-speed production networks.
UIC's National Center for Data Mining (NCDM) and Laboratory for Advanced Computing flashed a set of astronomical data across the Atlantic at 6.8 gigabits per second --- 6800 times faster than the 1 megabit per second effective speed that connects most companies to the internet.
In the test, 1.4 terabytes of astronomical data was transmitted from Chicago to Amsterdam in 30 minutes using UDT, a new protocol developed by the NCDM at the University of Illinois at Chicago. In comparison, moving the same amount of data using the TCP Protocol, which is the standard used on the internet today for data transfers, would take 25 days.
October 30, 2002: Data transfer demo sets speed markResearchers from Northwestern University and the University of Illinois at Chicago have set a new speed record for transmitting data: 2.8 gigabits, or billion bits, per second. The researchers set the record while transmitting information between Amsterdam and Chicago at the iGRID 2002 conference in Amsterdam on September 24.