Inclusive & Assistive Products
ISDN2001/2002: Second Year Design Project
Raspberry transmits information from its system to the LCD screen by SPI(Serial Peripheral interface) communication protocol.SPI communication protocol is frequently used in embedded systems for its strengths.[1] This kind of protocol consists of two parts, a master device and several slave devices. Each device has four data lines, which are:
- MISO(Master In Slave Out): MISO works with CS as a channel to transmit data from slave device to master device.
- MOSI(Master Out Slave In): MOSI works the opposite from MISO.
- SCK(Serial Clock): The SCK line synchronizes the time and speed of the data transmission.
- CS/SS(Slave Select): CS/SS line works as a selector to let the master device choose which slave device to communicate with.
The advantages of SPI are its simple implements, low pin count and Full-Duplex Communication. Except for the lcd screen, SPI protocol is also used in SD cards, IMU, etc.Also, the state of the button on screen can be read by raspberry pi through GPIO(General Purpose Input/Output (GPIO)).And the audio and camera information could be transmitted to raspberry by USB(Universal Serial Bus).
USB protocol is also a common communication way in daily life. The pins of type A USB are [2]:
- Pin1 (VBUS): Used for providing power supply.
- Pin2 (D-): It is a differential pair pin available in white color, used for connectivity of USB.
- Pin3 (D+): It is a differential pair pin available in green color, used for connectivity of USB.
- Pin4 (GND): It is a Ground pin
Both the camera and microphone used are type A. USBprotocol simply works on polling to continuously check whether the device is ready for data transmission. The content of data transmitted are mainly four types: Token, Data, Handshake & Start of the Frame. This structure ensures accurate and high speed data transmission.
The advantages of USB transmission are its hot-pluggable and quick identification. This kind of protocol could make development much easier by direct use instead of doing much of configuration.
Software
Raspberry pi is developed based on Linux Debian Bullseye. And to use the peripherals, some dependencies need to be installed.
The LCD display uses open-source library fbcp-ili9341[3]. This library provides an efficient way to display the desktop to the LCD display and supports custom settings as well.One problem meet here is that this library does not support the Debian Bookworm system. SoThe GPIO could be used directly by the RPI.gpio library of python in raspberry pi.The USB microphone could be read by the pyaudio library in python, for the camera we could use open-cv library to take the photo.
One problem is that pyaudio may recognize the camera to be the audio input device and therefore cannot read the voice. To solve this problem, we traverse all the devices and get the correct device by name, and get its device ID. And set the audio input to the correct ID.The UI could be constructed by curses, which is a simple text UI library in python. And it can be made like an 8-bit game.
Additional thing can be made is the cutscene. For camera taking part, a camera could be shown to indicate the next step is taking photos. For the audio recording part, a shaking bar could be used to indicate that it is recording now.
And the interactive logic needs to be restricted to three buttons. Two buttons on the right and one button the upper. All the interaction could be done by the three buttons.
After one process is finished, the device will store the image and audio to a local SD card, and send to the larger display devices after they are connected. Now it will directly send the file to remote server for next step.
Resources
Source Code:
https://github.com/Pledoge/ISDN2002-Photographer.git
References:
-
N. Singh, “Deep Dive into SPI protocol,” ThinkRobotics.com, Jun. 13, 2024. https://thinkrobotics.com/blogs/learn/deep-dive-into-spi-protocol?srsltid=AfmBOop6GN5Dh1ZdQLsQ8IN5_4wQXvrY-tyyTyAuigKMGMwGb3iR2jIi
-
T. Agarwal, “USB Protocol : architecture, working specifications & Applications,” ElProCus - Electronic Projects for Engineering Students, Jul. 18, 2022. https://www.elprocus.com/usb-protocol/
-
“juj/fbcp-ili9341: A blazing fast display driver for SPI-based LCD displays for Raspberry Pi A, B, 2, 3, 4 and Zero,” GitHub. https://github.com/juj/fbcp-ili9341