The Application Layer - Espressif Systems ESP8266 Beginner's Manual

MAC addresses and ARP (optional)
It is actually impossible to send packets directly to another machine using only the IP address. To send a packet to a specific device on
the LAN (Wi-Fi or Ethernet), you have to know its MAC-address. The MAC address is a unique number that is unique for every network
device, and it never changes, it's hardwired in the network chip. This means that every ESP8266, every network card, every
smartphone ... ever made, has a different MAC address.
So before the ESP can send a packet to your smartphone for example, it has to know its MAC address. It doesn't know this yet, the ESP
only knows the IP address of the smartphone, say 192.168.1.6. To do this, the ESP sends a broadcast message (i.e. a message
addressed to all devices on the LAN) saying "I'm looking for the MAC address of the device with the IP address 192.168.1.6". The ESP
also includes its own IP and MAC address with the message. When the smartphone receives this broadcast message, it recognizes its
own IP address, and responds to the ESP by sending its own MAC address. Now the ESP and the phone both know each other's IP and
MAC addresses, and they can communicate using IP addresses. This method is called the Addres Resolution Protocol, or ARP.
What about the Internet?
As you might have noticed, I only talked about the local area network, these are the computers in your own house. So how can the
ESP8266 communicate with the Internet, you may ask? Well, there's a lot of network infrastructure involved in 'The Internet', and they
all obey the IP rules, to make sure most of your packets arrive at there destination. It's not that simple of course, there's a lot of things
going on, like routing and Network Address Translation (NAT), but that falls outside the scope of this article, and it's not really
something most people have to worry about.
TL;DR
The Internet layer uses IP addresses in order to know where it should send the data. This means that two devices can now send
packets of data to each other, even over the Internet.
The Transport layer
The different devices in the network do their best to deliver these IP packets to the addressee, however, it's not uncommon for a
packet to get lost, so it will never arrive. Or the packet might get corrupted on the way: the data is no longer correct. IP also can't
guarantee that the packets arrive in the same order they were sent in.
This means that we can't reliably send messages yet by only using the link and the Internet layer, since we can never know when and
whether a packet will arrive, or know for certain that a received packet is correct.
We need a third layer on top of the Internet layer: the Transport layer.
There are mainly two protocols that make up this third layer: the Transmission Control Protocol (TCP) and the User Datagram Protocol
(UDP).
TCP makes sure that all packets are received, that the packets are in order, and that corrupted packets are re-sent. This means
that it can be used for communication between multiple applications, without having to worry about data integrity or packet
loss. This is why it's used for things like downloading webpages, sending email, uploading files etc.
UDP on the other hand, doesn't guarantee that every packet reaches its destination, it does check for errors however, but
when it finds one, it just destroys the packet, without re-sending it. This means that it's not as reliable as TCP, but it's faster,
and has a much lower latency, because it doesn't require an open connection to send messages, like TCP does. That's why it's
used in voice and video chats, and for example in online games.
If you want to know more about the differences between TCP and UDP, check out this video.
TL;DR
The IP protocol is not reliable, and has no error checking. TCP solves this by re-sending lost or corrupt packages, and orders packets
that are received in the wrong order. UDP also checks for corrupt packages, but doesn't re-send them, so it has less latency than TCP.
The Application layer
We now have reliable communication using TCP, but there's still one problem. Think of it this way: you are sending a letter, and TCP
guarantees that it will arrive at its destination, but if the receiver doesn't understand the language it's written in, he won't know what
to do with it.
In other words, we need a fourth layer of protocols, for two programs to be able to communicate with each other.
There's lots of different protocols out there, but we'll mostly focus on the protocols for web servers and browsers.
HyperText Transfer Protocol
The HyperText Transfer Protocol, or HTTP, is the protocol (cfr. language) that is used by both web servers and web clients in order to
communicate. It uses text to perform send requests and responses from the client to the server and back again.
For example, when you type http://www.google.com into the address bar of a web browser (client), it will send an HTTP GET request to
the Google web server. The server understands this HTTP request, and will send the Google webpage as a response. Or when you
upload an image to Instagram, your browser sends an HTTP POST request with your selfie attached to the Instagram server. The
server understands the request, saves the image and adds it into the database, sends the URL of the new image back to your browser,
and the browser will add the image on the webpage.
As you can see, neither the client nor the server has to worry about the integrity of the messages they send, and they know that the
recipient understands their language, and that it will know what to do with a certain HTTP request.
Most modern sites use a secure version of HTTP, called HTTPS. This secure connection encrypts the data, for security reasons. (You
don't want anyone reading the packets from your mail server, or the packets you sent to your bank, for instance.)
WebSocket
HTTP is great for things like downloading webpages, uploading photos etc. but it's quite slow: every time you send an HTTP request,
you have to start a new TCP connection to the server, then send your request, wait for the server to respond, and download the
response. Wouldn't it be great if we didn't have to open a new connection every time we want to send some data, and if we could
send and receive data at the same time at any moment we'd like? That's where WebSocket comes to the rescue: you can keep the TCP
connection with the server open at all times, you get perfect TCP reliability, and it's pretty fast.
Open Sound Control