This final blog post on file uploads focuses on web sockets, the latest approach to browser/server communication.
Part 1 of this series reviewed the history of file uploads including the original specifications, as well as the client-side code to upload a file. Part 2 presented asynchronous file uploads and focused on improvements to the XMLHttpRequest object, along with additional user experience enhancements such as drag and drop file uploads and progress bar indicators.
As the web continues to change and grow, the limitations of unidirectional communication from the browser to the web server has become apparent. Therefore, a new standard allowing bi-directional communication has emerged. Enter, web sockets.
Performing a web sockets-based file upload requires several steps. First, APIs are needed to load the file from the file system, then the file is transformed into a suitable payload to be sent over web sockets, and finally, server-side code is required to receive the file.
In previous posts from this series, the server side mechanism was not discussed. This is because all web server-based platform readily handle file uploads, but for a web sockets based approach, some extra configuration is required.
In this article, we will examine two possible ways to send a file over web sockets. The first implementation will be a pure web sockets approach, and the second will use the Socket.IO library.
A Little Known API: File Reader
The existence of the File Reader API is known to very few web developers. However, it is available in all modern browsers, and is officially part of Internet Explorer (IE) since IE 10. Files selected with a File Input element, or dropped using the Drag and Drop API, can be read with the File Reader object from the user’s local file system.
On the surface, this may appear to be a bit unsecure, possibly even violating the sandboxing policy of web browsers and how they allow web pages to interact with the local system resources. However, since the File Reader is limited by the browser to only read files selected through the ordinary means of file uploading, it is not possible for the web page to arbitrarily load a file. Additionally, the File Reader object does not permit writing to the client file system, as the name suggests. Finally, the File Reader object cannot be used to browse the local file system.
Below is sample code that demonstrates how to use the File Reader object in conjunction with the drop event of a Drag & Drop file upload.
dropZone.addEventListener("drop", function(e) {
// prevent browser default behavior on drop
e.preventDefault();
// iterate over the files dragged on to the browser
for (var x=0; x < e.dataTransfer.files.length; x++) {
// instantiate a new FileReader object
var fr = new FileReader();
// loading files from the file system is an asynchronous
// operation, run this function when the loading process
// is complete
fr.addEventListener("loadend", function() {
// send the file over web sockets
ws.send(fr.result);
});
// load the file into an array buffer
fr.readAsArrayBuffer(file);
}
});
Binary the JavaScript Way: Array Buffers
Once the File Reader object loads the file, the contents of the file can be stored in several formats: Array Buffer, Binary String, Data URL, or Text. Discussing these various formats is beyond the scope of this post, but suffice it to say, web browsers have become very capable when it comes to working with binary data. The focus for file uploads is the Array Buffer because it works very well for sending binary data over a web socket. When used in conjunction with a Node.js web socket server, the Array Buffer object translates readily to a Node.js Buffer object.
What are Array Buffers?
Array Buffers are used to hold a fixed amount of binary data. The binary data can represent anything, i.e., it is a general purpose container for raw binary data. Array Buffers are unique structures. Once they are loaded with data, they are not directly manipulated by JavaScript code. Instead, the code interacts with Array Buffers through typed arrays and Data Views. To perform File Uploads, we only need to pass the Array Buffer returned from the File Reader to the Web Sockets send function. Array Buffers is a much larger discussion, but they can be especially useful when working with binary data in the web browser.
The code below is from the first code snippet from above. This method readAsArrayBuffer loads the file into an Array Buffer object, and makes the Array Buffer available through the result property on the instantiated File Reader Object (as shown in the loadend event handler below).
// load the file into an array buffer
fr.readAsArrayBuffer(file)
fr.addEventListener("loadend", function() {
// send the file over web sockets
ws.send(fr.result);
});
Web Sockets: Bi-Directional Communication
When the web was first created and web browsers were implemented, the only method of communication between the web browser and the web server was for the web browser to make a request to which the web server replied with a response. As web applications became more sophisticated, it became possible to request data from the web server after the initial page load. This lead to the eventual creation of Single Page Applications (SPAs). After requesting only the data with JavaScript, it would be processed instead of directly rendering on the screen.” This asynchronous loading of data was accomplished using techniques such as hidden iframes, and then later the XMLHttpRequest object.
The primary limitation of this method of communication is that the client always had to make a request first before the server could send data – it was unidirectional. While unidirectional communication was sufficient for many web site needs, the ability to continuously stream real time information was not possible. To work around this problem, the technique of long-polling was used to maintain an open HTTP connection between the web browser and the web server for an extended period of time. Through this extended connection, the web server could continuously respond to the original request, and send new data when it chose to. While helpful, this technique was really a hack, and limited the number of new requests a web server could handle. A better solution was needed, and that solution was web sockets.
Web Sockets is a Web Browser API in HTML5. Web Sockets permit true bi-directional communication between the web server and the web browser. While Web Sockets are not the same as HTTP, they do sit on top of HTTP, and are initiated via an upgrade request over HTTP. The client must first initiate the connection, but once established, both the web server and web browser can initiate data transfers. The connection can stay open during the life of the user’s session on the web site without impacting web server connection availability.
The immediate practical usage of web sockets was to send real-time streaming data, such as news stories or stock quotes, from the server to the web browser. However, they could be used for so much more. New JavaScript frameworks such as Meteor used web sockets not just for real-time streaming from the server, but also for sending all kinds of data back and forth between the web server and the web browser as an alternative to AJAX calls.
Initially, web sockets were only supported in the latest browsers; therefore, socket libraries such as Socket.IO for Node.js and SignalR for .Net were used to allow for graceful fall backs to long polling for web servers and web browsers that did not support web sockets. As web sockets became the norm and not the exception, they became increasing useful for file upload operations. Combined with Array Buffer objects, they allow not only text data but binary data to be transmitted.
The code below demonstrates how to pass an Array Buffer through the web sockets send function.
fr.addEventListener("loadend", function() {
// send the file over web sockets
ws.send(fr.result);
});
Socket.IO: Making Web Sockets Even Easier and More Capable
When building real applications, developers rarely use the Web Socket object directly. Similar to the XMLHttpRequest object, developers typically use another library built on those technologies for two primary reasons. First, while web sockets support is the norm now, there is still a substantial portion of users who have older versions of Internet Explorer. Internet Explorer 9 and earlier do not support web sockets, so they must fall back to the long polling approach. Also, there are many use cases involving web sockets that require a significant amount of programming in order to use the Web Socket object directly. Such cases include broadcasting to all clients from the server, broadcasting to groups of clients from the server, and implementing custom events.
To support these more complex use-cases, developers use libraries like Socket.IO to provide an easy-to -use interface, while hiding the complexities of managing multiple client web socket connections. This method can also map various web socket transmissions to user-defined events. Using Socket.IO to send files over web sockets is very similar to how it’s accomplished with HTML5 Web Socket objects. The code sample below shows how to configure Socket.IO both on the server and the client. For a full implementation please check out the links at the end of this blog post.
fr.addEventListener("loadend", function() {
// emit a custom event we have named 'upload file'
// In Socket.IO, emit sends data to the web socket server
socket.emit("uploadfile", {
fileData: fr.result
});
});
Conclusion
Web sockets are extremely useful for performing file uploads. Using the file objects created by the File Input and Drag & Drop APIs, the File Reader object can be used to read local files into Array Buffers, which are then sent over web sockets to the web server. Furthermore, using sophisticated libraries such as Socket.IO allows for the easy creation of complex web applications. These apps not only support file uploads over sockets, but also allow for the creation of large-scale, bi-directional communication-driven web applications.
This concludes our three part series on performing file uploads with web browsers. Over the 20-year history of this capability, much has changed and improved. Today, Internet users around the world leverage web-based file storage to manage and access their data.
Eric is a professional software developer specializing in HTML, CSS, and JavaScript technologies. He has been developing software and delivering training classes for nearly 19 years. He holds the MCSD Certification for ASP.Net Web Applications, and is a Microsoft Certified Trainer.
