In this article, a DIY wizard will tell us how to make your own smart glasses using RaspberryPi. The idea to make such a project came to him after watching Spider-Man.
Before he took on this project, the most advanced thing he could do was blink an LED with an Arduino UNO or print “Hello World” in Python. One can imagine how difficult the project was for him. He has no professional education in electrical engineering, despite this, he finished the project and through it he learned a lot about electronics.
The features of these smart glasses are as follows:
Uploading Photos/Videos to Dropbox
Calculating Math Problems
Tools and materials: -Transparent graphic OLED screen; -Charging module; -Raspberry Pi Zero; -Lithium polymer battery; -PiCamera; -Wireless headphones; -Soldering accessories; -3D printer; -Film for printer; Step one: prototype
This is the second version of the glasses, and he made his first prototype of glasses over a year ago. The main feature of these smart glasses was to take pictures and perform basic text recognition, and then provide the user with some information about it using Wolfram (a question and answer site powered by artificial intelligence technology). For example, if he photographed, say, a detour sign, the glasses would recognize the text and give the user a definition. However, these glasses did not work quite correctly, and the text recognition had a very long delay. As for the new version, the wizard decided that this feature is not very practical, and if the user needs this information, he can simply ask the chatbot. He decided that version 2 should focus more on reliability and efficiency, while adding more features.
Step two : Components
Raspberry Pi Zero
The master chose the Raspberry Pi Zero as the brain for this device. The Raspberry Pi Zero easily connects to the Internet, supports Bluetooth for wireless headphones, Python is pre-installed and has an easy-to-use camera interface. It has significantly more RAM, in particular the Raspberry Pi Zero has 520MB of RAM as opposed to the 512KB ESP32.
Sparkfun's clear OLED screen is a defining characteristic of smart glasses that gives users feedback. It can be controlled via I2C or SPI (I2C will be used in this project). The display has a minimum operating voltage of 1.65V and a maximum operating voltage of 3.3V, so it is not advisable to power it directly from the 5V pins. If you ever intend to use this display in another project, the driver comes with multiple QWIIC connectors that can connect to other compatible Sparkfun boards.
This device needs a charging circuit that could boost the voltage as well from 3.3 to 5 V. The board used in the project is very easy to connect.
After examining the recommended power supply requirements for RaspberryPi, the wizard found this diagram on the official RaspberryPi website. She says the recommended power supply should be 1200mAh or 1.2A. Considering that the transparent OLED display consumes only 400 mAh and the camera is 250 mAh, this should be more than enough.
Step Three: Required Applications and API
* Applications Required:
PuTTY is the software that will be used to ssh (an application layer network protocol that allows remote control of the operating system) in RaspberryPI before accessing the GUI desktop. To download it, click the link and click here. Next you will be taken to a new page where you can install the 32-bit or 64-bit version.
You will need BalenaEtcher to flash Raspbian OS to SD card. Click the link to go to their website where you can choose your operating system and 32-bit or 64-bit version.
Microsoft Classic Remote Desktop
Classic Remote Desktop is the way to go to the RaspberryPI GUI. Classic Remote Desktop is available for download from the Microsoft Store. If you are not using a Mac, a possible alternative is Chrome Remote Desktop.
Angry IP Scanner
Once the RaspberryPI is connected to the Internet, you will need to access it via its IP address. Angry IP Scanner scans all devices on the user's wireless network. To install it, follow the link and click “free download”, then “32/64-bit Installer”. This installer will automatically detect which 32-bit or 64-bit system you have.
Cura is the encoding software needed to convert stl files to gcode files, which will then be uploaded to a 3D printer. When visiting their website, you need to click the download button and select your operating system. Cura is only available on 64-bit systems, but there are alternatives like Slic3r.
API (Application Programming Interface) you need to get it going:
Dropbox is a server used to store any kind of information, essentially a cloud storage. It will be used to store images and videos captured by smart glasses so that they can be accessed from any device. To download Dropbox, follow the link above and register or sign in. When you're done, hit app console – & gt; 'create app' – & gt; 'scoped access' – & gt; 'app folder', then choose a name for your application and click finish. After you finish all of this, you should see the control panel for your application. Leave this tab, we will come back to it later.
One of the many functions of the smart glasses is sending SMS or text messages. To accomplish this task, the wizard will use Twillio, which is “free” to use an API capable of sending or receiving SMS or calls. The reason “free” in parentheses is that this service is free only until the trial balance of $ 15 provided is depleted. Sending an SMS costs a little over one cent per message. That's about five hundred messages.
The setup process is simple. Go to the website and register for a free trial account. After confirming your phone number, Twilio asks a few questions. Scroll through the first two questions, then select with code, then Python and finally No, I want to use my own hosting service. You are now in your console, which will display everything and everyone about your account and your apps. Click “Get a trial number”. This is your phone number that Twilio will use to send SMS. For a trial account, you need to confirm the numbers to which we would like to send SMS. To do this, go to the console and on the right you will see the “Verified Numbers” link, your number will already be recorded there, but if you want to write to someone else, you will need to add their phone number here. Don't close the console just yet.
Last but not least is Wolfram Alpha, a computational search engine. Wolfram Alpha will be the brain of smart glasses. Wolfram Alpha allows us to get answers to any question we may ask or a math problem we propose. Signing up is easy, just follow the link and sign up for an account, then click your profile in the top right corner and select “My Apps (API)”. Now click on “Get AppID” where it will ask you for some description of your application and then finally it will give you your application ID. Don't close this screen, we'll come back to it later in the code.
* Some functions may not work in Russia.
Step Four: Configuring RaspberryPI
Operating System Firmware
First, install Raspbian OS from the official site. The wizard uses BalenaEtcher to flash the OS to a micro-SD card. Connect the micro-SD card to your computer via an adapter, then open BalenaEtcher. Select the .zip folder containing the OS, then select the appropriate SD card and press flash, it will take about 20 minutes. When the download is finished, disconnect the SD card.
Connecting RaspberryPI to Your Wi-Fi
Now that the OS image is written to the SD card, you need to add a couple of files that will allow you to get Wi-Fi access to the RaspberryPI. Insert the SD card back into your computer and navigate to the appropriate directory in File Explorer or Finder. Create an empty text file named ssh, but remove the filename extension to get a file named ssh. If you don't see the .txt at the end, you may need to include file name extensions. To do this, open File Explorer, then click View at the top of the window, and then select the File Name Extensions check box in the upper right corner.
The following file will be used by RaspberryPi to find out which network the user owns and how to connect to it. Create a new text file in your SD card directory named wpa_supplicant.txt. Paste the following lines of text into this file and replace “WIFI_SSID” with your network name and “WIFI_PASSWORD” with your network password, then save it. Now replace the txt file extension with conf.
country = US ctrl_interface = DIR =/var/run/wpa_supplicant Group = netdev update_config = 1 network = & # 123; ssid = & # 34; WIFI_SSID & # 34; scan_ssid = 1 psk = & # 34; WIFI_PASSWORD & # 34; key_mgmt = WPA-PSK}
Step five : SSH to RaspberryPI
Now that the RaspberryPI is ready to go online, turn it on by connecting the micro-usb port labeled “PWR” to your laptop. Please note that the RaspberryPI is a 2.4 GHz device. This means that if you only have a 5GHz network, you will not be able to connect to the RaspberryPI wirelessly. Most networks, however, are 2.4 GHz and 5 GHz. Then you will need to find out the IP address of the Raspberry PI by scanning all the devices on the network. Open Angry IP Scanner and scan your network by clicking the start button in the top right corner. Your RaspberryPI should have the hostname raspberrypi, so find that device in the list and write down the IP address. Now that you have an IP address, open PuTTY.
Next, a window will appear that says “log-in”. The default username is pi and the password is raspberry.
As with any new RaspberryPI, you need to make some updates first. Enter these commands in a terminal one at a time.
sudo apt-get update sudo apt-get upgrade sudo apt-get autoremove sudo reboot
Enabling the GUI
Before enabling the GUI, you need to change the hostname. Enter sudo raspi-config and a new window will open. Using the arrow keys, go to the second option and press Enter, then enter the new hostname. Without leaving this window, you also need to enable the SSH server, so go to “Interfacing options”, then scroll down to the place where it says SSH, press Enter. Click finish and reboot. SSH is now enabled and ready to use.
View the GUI
You can now access the GUI. Close this Putty session and go to remote desktop. Click “add” in the upper right corner, then “Desktop” and enter the IP address of the RaspberryPI. While RaspberryPI is enabled, click the IP address tab and the device will wirelessly access the GUI desktop.
Step six: download packages and music
Even though Python is preinstalled on RaspberryPI, you still need to install some of the libraries needed for the glasses. Below are the commands to install all packages not included in the initial Python installation. Enter them one by one into the shell editor.
pip install bs4 pip install urllib3 pip install luma.core pip install luma.oled pip install wolframalpha pip install SpeechRecognition pip install PyAudio pip install wikipedia pip install picamera pip install dropbox pip install twilio
Now we need to upload some music files to the RaspberryPI. First download this song and convert it to wav format with an online converter. Now rename it to “Imagine.mp3” and copy it to your RaspberryPi desktop. Although you can download any song you like, just make sure you save it to your desktop in wav format and change the name in the code to match the file name.
Step Seven: Connect Wireless Headphones
To receive data from the user and provide feedback, you need to connect wireless headphones to the RaspberryPI. There are several ways to do this, and the wizard will tell you about two. If the first method doesn't work, you can always try the second.
This method is by far the easiest. Please make sure your bluetooth headphones are in pairing mode first. Next, click the Bluetooth icon in the upper right corner of the screen and select the tab labeled GS. If the devices are connected, you should see a green checkmark in this menu and the LED should stop flashing.
First you need to initialize the Bluetooth agent. Enter these commands one at a time into the shell editor.
bluetoothctl power on agent on default-agent
Put the headphones into pairing mode by holding down the power button. Now enter this command.
During the scan, you should see the MAC address of your headset. If you do not know what your device has a MAC address, the wizard recommends connecting it first to the phone, where it should be displayed in the settings of the wireless device. Once you see your MAC address, enter the command below.
Next, enter the commands below into the editor, replacing the asterisk with the MAC address your device.
pair * trust * connect *
Step eight: text-speech
Smart glasses will transmit information mainly through a chatbot. The chatbot will work with what is called a text-to-speech library, which simply takes a regular input string or text and converts it to an MP3 file. Initially, the wizard wanted to use the pyttsx3 library to control the chatbot, but it didn't work. After a little research, he discovered another speech engine, Espeak.
To load it, open an editor and enter the following command.
sudo apt-get install espeak
espeak & # 34; Hello World & # 34; -ven + f3 -k5 -s125 --stdout | aplay -D bluealsa & # 58; DEV = XX & # 58; XX & # 58; XX & # 58; XX & # 58; XX & # 58; XX, PROFILE = sco & lt; br & gt;
Note: wherever you see “XX: XX: XX: XX: XX: XX”, replace it with the MAC address of your wireless headset.
By itself, Espeak has a rather robotic sound, but has a number of settings for tone, gender, speed and much more. Below are a few modifications you can make, but if you'd like a complete list, follow this link.
-ven ~ Specifies the language. Change en to the abbreviation for the language you want.
-f ~ reads the text file
-a ~ Set amplitude, range 0-200.
-s ~ Changes speed. The lower limit is 80.
-g ~ Increases the pause between words by 10ms.
-f3 ~ Sets the third female voice. There are ten options, five for men and five for women. (for example, m1, f2 …)
Step nine: photo video
Another great feature of the smart glasses is video and photo shooting. Taking pictures with RaspberryPI is very easy. All you have to do is connect the camera to the ribbon cable connector. To take a photo, you need to turn on the camera. Click on the RaspberryPI icon in the upper left corner and scroll down to preferences-> Raspberry PI Configuration-> Interfaces. There is a checkbox for connecting a camera in the menu. Next, you need to enter the sudo reboot command and reboot.
Next, open a shell editor and enter the following command:
raspistill -o Image.jpg
After pressing the enter button, the camera LED should light up for a few seconds, after which the image named Image.jpg will be saved in the/home/pi directory.
Step ten: screen
Finally, we got to one of the coolest parts of smart glasses – the transparent OLED screen. It will display information for the user such as time, temperature and just general information. As stated earlier, this display will work through I2C and the python luma.core library. I2C is not enabled by default, so you need to enable it. Go to the same place where you turned on the camera module, turn on the I2C and reboot.
Now you need to solder the wires from the display to the RaspberryPi. The wizard recommends a soldering temperature of 725 ° F/385 ° C. Remember to turn off your RaspberryPi and disconnect all peripherals.
OLED — RaspberryPI
Step eleven : loading the code
Now that the RaspberryPI, transparent display and camera are set up and working, you can start code. Below is the annotation and the code itself.
# 1 Importing Libraries
These are the libraries we installed earlier in this tutorial, so all you have to do is copy the code.
# 2 AppID and Authorization Tokens < br>This is where all our App_IDs and account tokens are, as well as some other variables. Replace contact numbers and names with people you want to send SMS to. Place all of your Application IDs and Authentication Tokens from Step 3 in the locations shown.
# 3 Functions
In this section, we define many tasks to display that would be rather tedious to type every time. This is also where the startup sequence is defined and initiated.
# 4 The while and if statements
This is a cyclical piece of code that repeats endlessly until you issue the “exit program” command. After examining this section in more detail, we will see that glasses can inform you of news, send SMS, determine the time, take pictures, shoot videos and play music based on the trigger words inside the if-statement.
# 5 Search
If if you haven't said a single trigger word, the program will look for everything you said in Wolfram, but if it fails, it will search Wikipedia and give a summary.
Make sure your wireless headphones are connected before attempting the code. If all goes well, the display should turn on and the chatbot will ask what you want from it. He will listen to your answer whenever there are three dots at the bottom of the screen. Below is a complete list of commands. When you're done, just say “exit program” and press the power switch.
news – the chatbot will give you three main headlines from Google News.
exit program – will stop the script and then all you have to do is flip the switch.
SMS – any message will be sent to whoever you tell.
Time – time.
Picture – will take a picture and upload it to your Dropbox account.
Play my music – the first song on the list will play, and when finished you will be asked if you want to play the next one.
Video – will record 30 seconds of video, then convert it from h264 format to MP4 format.
Step twelve: setting up the power supply
This step requires a soldering iron, solder, charging module, slide switch, and battery. Below are the connections.
Adafruit 500c – Slide Switch
en – side pin
gnd – middle pin
Adafruit 500c – Li-Po
Bat – Positive
Gnd – Negative
Step Thirteen: 3D Printing
The 3D printed glasses are made up of 3 parts: the left earpiece, the right earphone and the front. Filament for printing – PLA. If you don't have access to a 3D printer, cardboard or MDF can be used for the case.
Printing temperature 215
Bed Temperature 70
Speed 100 mm/s
Layer height 0.12 mm
Initial speed 20 mm/s
Print files .
Step thirteenth : build
The wizard now has three different parts: the battery diagram, the RaspberryPI, and the 3D parts. Before connecting the battery to the RaspberryPI, you need to pass the wires through the plugs. First, solder the wires to the 5V pin and GND pin of the Adafruit 500c. Now pass these wires through the holes in both covers and then to the RaspberryPi power input pins. Make sure the slotted cover is on the right. Below are the connections for the schematic.
Adafruit 500c – RaspberryPI
5V – 5V (pin 2)
GND – GND (pin 6)
Next, align the Adafruit 500c charging port with the hole on the bottom of the left earbud and glue the bottom to the case with hot melt glue. Place the battery next to the charger, then slide the cover down the wires and hot glue it. The left earphone is ready, now we will deal with the right one. Place the RaspberryPI in the right earbud and slide it all the way back, if it is not completely pushed back the camera will not fit. Wiggle a little and slide the camera back into place. Disconnect the clear OLED display from the cable. Pass the ribbon cable through the slot in the cover, and then reinsert it. Pull the cover down and stick it in place. If there are glasses, glue them in place. All that's left to do is connect the two earbuds in front.
Step fourteen: test
Now we need to test smart glasses. First, turn on your RaspberryPi by flipping the power switch. You should see a dim green and red light inside the right earcup. Next, turn on the wireless headphones, which should connect automatically. Finally, open a shell editor and enter the command:
This command will run the script. After a short startup delay, the transparent OLED display will turn on and the chatbot will start talking. Feel free to ask him some test questions like what news, what time, send SMS or turn on my music!
Step Fifteen: Troubleshooting
Problems with xrdp Installation
This issue can occur if you've tried to install it before. To fix, open a shell editor and type:
sudo apt-get remove xrdp vnc4server tightvncserver
If you were unable to take pictures with using the “raspistill” command, there may be no changes made to the raspi-config. Check it out by clicking RaspberryPI in the upper left corner and go to preferences-> Raspberry PI Configuration-> Interfaces. Make sure the checkbox next to the camera that says “enable” is checked. Then go to your shell editor and type sudo reboot.
Everything is ready.