Video Streaming using Python
In this article, I am going to explain how we can create a video chat app using Python OpenCV and Socket Programming.
For this practical, I am going to use Python OpenCV and Socket Programming so let’s first understand Python OpenCV and Socket Programming.
Python OpenCV
OpenCV-Python is a library of Python bindings designed to solve computer vision problems. OpenCV-Python is a Python wrapper for the original OpenCV C++ implementation.
OpenCV-Python makes use of Numpy, which is a highly optimized library for numerical operations with a MATLAB-style syntax. All the OpenCV array structures are converted to and from Numpy arrays. This also makes it easier to integrate with other libraries that use Numpy such as SciPy and Matplotlib.
Socket Programming in Python
Socket programming is a way of connecting two nodes on a network to communicate with each other. One socket(node) listens on a particular port at an IP, while the other socket reaches out to the other to form a connection. The server forms the listener socket while the client reaches out to the server.
Socket programming is started by importing the socket library and making a simple socket.
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
Here we have created a socket with two parameters. The first parameter is AF_INET and the second one is SOCK_STREAM. AF_INET refers to the address-family ipv4 and SOCK_STREAM means connection-oriented TCP protocol.
What is a Network Socket?
A network socket is a software structure within a network node that serves as an endpoint to send and receive data. Its properties are defined by the network API. It exists during the process of application and externally identified by its socket address.
A socket address is the combination of protocol type, IP address, and Port number for data communication. A remote process establishes a socket in its connection to its protocol stack and then uses network API to connect to the application. It presents its own socket address for use.
In standard internet protocols like TCP and UDP socket address is the combination of socket address = (IP address, port number)
Types of Internet Sockets:
- Stream sockets: Stream sockets allow processes to communicate using TCP. A stream socket provides a bidirectional, reliable, sequenced, and unduplicated flow of data with no record boundaries.
- Datagram sockets: Datagram sockets allow processes to use UDP to communicate. A datagram socket supports a bidirectional flow of messages. A process on a datagram socket can receive messages in a different order from the sending sequence and can receive duplicate messages.
- Raw sockets: Raw sockets provide access to ICMP. These sockets are normally datagram-oriented, although their exact characteristics are dependent on the interface provided by the protocol. Raw sockets are not for most applications.
Client-Server Model in Network Programming
At first, the server will create the socket and will go to the listening mode. Initially, the server will not require the IP address of the client but the client will know the IP address and port number of the server to connect.
Video Chat App using Python
Just like text chat apps, video chat app provides a way for users to communicate via live video streaming. Video Chat App helps users to communicate with the face-to-face connection between two users via the internet.
For creating Video Chat App using Python I am going to create two files in python one is for the client-side and the other is for the server-side. I will use the stream socket where the server code with extracting the video from the server webcam and then it will send that video to the client using socket programming.
Server-side Python Code:
First, we will create the socket by specifying the address family and address type to the socket.socket() function.
server_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
Then I am going to bind the socket with the Host Ip address and with some port number. The port should be an integer value between 1000 to 6265535.
server_socket.bind(socket_address)
Then I am going to use listen to function to accept the connection. Here I am giving backlog as 5 which means five connections the system will allow. Also, I am using accept function the accept the connection and getting the client socket and the address.
server_socket.listen(5)
client_socket,addr = server_socket.accept()
After that, it can handle the client socket to accept and send the message using send function, and finally, we can close the socket with the close function.
client_socket.recv()
client_socket.sendall()
client_socket.close()
At the server-side I am using OpenCV to get the frames of the webcam then each frame data will be serialized to bytes data using the pickle module and then it will be packed using the struck module.
Here is the complete code for chat-server.py
# This code is for the server
# Lets import the libraries
import socket, cv2, pickle,struct,imutils
# Socket Create
server_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
host_name = socket.gethostname()
host_ip = socket.gethostbyname(host_name)
print('HOST IP:',host_ip)
port = 9999
socket_address = (host_ip,port)
# Socket Bind
server_socket.bind(socket_address)
# Socket Listen
server_socket.listen(5)
print("LISTENING AT:",socket_address)
# Socket Accept
while True:
client_socket,addr = server_socket.accept()
print('GOT CONNECTION FROM:',addr)
if client_socket:
vid = cv2.VideoCapture(0)
while(vid.isOpened()):
img,frame = vid.read()
frame = imutils.resize(frame,width=320)
a = pickle.dumps(frame)
message = struct.pack("Q",len(a))+a
client_socket.sendall(message)
cv2.imshow('TRANSMITTING VIDEO',frame)
if cv2.waitKey(1) == '13':
client_socket.close()Client-side Python Code:
First, we will create the socket by specifying the address family and address type to the socket.socket() function.
server_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
Now I will use the connect function to connect to the server socket. The host IP address and the port number will be the same as the server.
client_socket.connect((host_ip,port))
Now we can send and receive the information using send() and recv() function. Here we will use 1024 as buffer size as TCP is a streaming protocol which means there are no message boundaries it is just the stream of message bytes. It is like FIFO and when the server will send 1024 bytes then 1024 bytes will be received by the client.
packet = client_socket.recv(1024)
client_socket.send(packet)
The socket can be close the socket with the close function.
client_socket.close()
Here at the client-side first, we will unpack the received packet data using struct module and then will load the frame using pickle, and finally, we will display the received video.
Here is the complete code for client-server.py
import socket,cv2, pickle,struct
# create socket
client_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
host_ip = '192.168.1.20' # paste your server ip address here
port = 9999
client_socket.connect((host_ip,port)) # a tuple
data = b""
payload_size = struct.calcsize("Q")
while True:
while len(data) < payload_size:
packet = client_socket.recv(4*1024) # 4K
if not packet: break
data+=packet
packed_msg_size = data[:payload_size]
data = data[payload_size:]
msg_size = struct.unpack("Q",packed_msg_size)[0]
while len(data) < msg_size:
data += client_socket.recv(4*1024)
frame_data = data[:msg_size]
data = data[msg_size:]
frame = pickle.loads(frame_data)
cv2.imshow("RECEIVING VIDEO",frame)
if cv2.waitKey(1) == '13':
break
client_socket.close()