Episode 51

Hello traveler. Today we are continuing our journey through abstraction layers in a web application with all its foundations. Two episodes ago we started from Java code written by hand, followed by the web framework. One episode ago we went down into web servers, traveled through Java virtual machine to finally meet the container engine. Finally? Oh no, there is still long way before us.

The cruise on the back of giant blue whale is over, and the Lonely Mountain is closer than ever. The Hobbit jumps out of the barrel and lands on solid ground somewhere in the city harbor. Meanwhile, the whale drifts away, unnoticed by busy workers. The hobbit leaves the dock and treads towards the building that dwarfs nearby ones with its size. The city hall.

On the Doorstep: The Operating system

In terms of the operating system, we are now crossing the border between the application and the kernel. We can say, that both sides were mediated by the container engine, which presents virtual kernel API to the application inside the container and maps all calls to the actual kernel API on which it resides. Java virtual machine and everything that’s sits on top of it, is now just another process, with its own snapshot of machine code to be executed, virtual memory, processor state, resource descriptors and other stuff. The virtual memory contains heap and stack areas and is a layer of abstraction over physical memory allowing for better address space continuity and allocation of additional memory. Mind that JVM addressing space is an abstraction over virtual memory here. Anyway, resources that are of our interest are file descriptors.

Why do we need file descriptors? Because network sockets are modeled as file descriptors in Linux. And in order to push our Hobbit forward, we need to establish a network connection using sockets. They sit between third and fourth layer in Open Systems Interconnection networking model, before Application layer, Presentation layer and Session layer. Application layer is our business logic. Presentation layer is somewhat vague, in case of HTTP protocol, it’s attributed to character encoding and optionally SSL encryption. Session layer is implemented in an HTTP protocol, which let us maintain conversation with our user, for example when browsing online shopping cart. In our case however, the Hobbit consumer (or, less grievous, the destination) is a machine, not human. We don’t really need a conversation here. Following the REST principles, we want to be stateless.

Meanwhile, let’s get back to sockets. In order to send the data, our process needs to perform a system call and asks the kernel to open file descriptor and create an appropriate socket. We can think of the socket, as a gateway to Transport layer, a further one in the networking model. We need to provide destination IP address and port, and…

The Hobbit leaves the city behind and hikes up and down through the hills and valleys on approach to the Lonely Mountain. Surprisingly, every now and then, more companions are joining in, some of them preferring to go in front of the fellowship, some of them trailing behind.

Inside Information: The Internet Protocol Suite

There are actually many networking models. The OSI model defines seven layers but the reference Internet Protocol Suite model (RFC 1122) defines only four. Three top layers of OSI – Application, Presentation and Session are just called Application in RFC 1122. Then layers go one to one with OSI: Transport, Network (or Internet) and Data Link. And finally, RFC 1122 omits the Physical layer. So shall we for now. In order to accomplish their goals, each layer adds its own header to original message.

The Transport layer, in our case implemented via TCP protocol, is used to establish and maintain connection with a process running on remote host, where the host (the particular machine) is identified by IP address, and the process – by port number. There are common ports for typical services like FTP or SMTP, but we are mostly interested in ports used by HTTP  servers: 80. Let’s get down to business now. The TCP protocol performs handshaking with desirable host, establish a session and, hopefully, we are ready to shove the Hobbit in. Note, that TCP session and HTTP session are two different concepts. HTTP session spans multiple HTTP request/responses, while TCP session encapsulates transmitting a single HTTP payload. In general, Transport layer sends data in packets and makes sure, they all arrive correctly at their destination in the right order, re sending them as needed and providing congestion control and operates on process level. But who knows how to deliver a single packet from host A to B?

The Network layer, typically implemented via IP protocol, handles individual packets and makes sure they reach the destination host. Network layer is oblivious of the higher level protocol. It can be TCP, but also UDP, DCCP, RSVP or other, who gives a damn. The job is to deliver a packet. Who knows, maybe if our Hobbit gets fat enough, he will have to be divided into two or more packets! Sounds scary, but no worries, at the end he will be reassembled and just fine. IP protocol performs routing, throwing the poor packets from one router to another across multiple networks of the Internet as it sees fit. It tries its best, but sometimes packets get lost. In that case, that’s the TCP problem to send the messenger again. But how pass the packet from intermediate device A to B?

The Link layer, implemented via protocols like PPP or Ethernet, takes care about establishing a direct connection between two physical network devices. While IP protocols cares about WAN network, or where to hop globally to deliver the message, Ethernet performs the hop in LAN – local network. Again, it is oblivious of the protocol above, the network topology and IP addresses. It’s just told to hop, so it hops. In order to do that, it needs to operate on physical network device addresses, namely MACs. It encapsulates higher level protocol packets into frames of bytes and, as the name suggest, it controls access to physical medium, be it electricity on the wire, light in fiber channel, radio waves or pigeons carrying letters written with dragon blood on parchment. We will get there.

Not at Home

Disclaimer: This episode was mostly written in a train from Wrocław to Gdańsk, a day before New Year’s Eve. Generously delayed to give me more time for research and proofreading. Also I arrived at the conclusion, that I will need a fourth part because no one will read that wall of text in one sitting.

The Hobbit is now accompanied by several new cheerful companions carrying different maps, instructions, shipment papers, letters of recommendation and mysterious notes. He is slowly walking on his toes down the tunnel into the depths of Lonely Mountain, as quietly as a mouse. There is faint light at the end, but after few more moments of uncertainty, the truth becomes obvious: The physical dragon is not there anymore.

Suddenly, in the distance, the mighty roar crumbles the sky upon the other side of the mountain. The roar is quickly fading away, towards the Lake City.

image source