SunFounder picar-x User manual

SunFounder picar-x
www.sunfounder.com
Feb 21, 2023

CONTENTS
1 Introduction 5
1.1 The History of Self-driving Cars ..................................... 5
1.2 About PiCar-X .............................................. 7
1.3 Deep Learning and Neural Networks .................................. 8
2 Component List and Assembly Instructions 9
3 About Robot HAT 11
4 Play with Python 13
4.1 Quick Guide on Python ......................................... 13
4.1.1 What Do We Need? ....................................... 13
4.1.2 Installing the OS ........................................ 15
4.1.3 Set up Your Raspberry Pi .................................... 22
4.1.4 Install All the Modules ..................................... 41
4.1.5 Enable I2C and Camera Interface ................................ 44
4.1.6 Servo Adjust .......................................... 48
4.2 Calibrating the PiCar-X ......................................... 50
4.3 Let PiCar-X Move ............................................ 53
4.4 Obstacle Avoidance ........................................... 55
4.5 Line Tracking .............................................. 57
4.6 Text to Speech .............................................. 58
4.7 Computer Vision ............................................. 60
4.8 Color Detection ............................................. 63
4.9 Face Detection .............................................. 68
4.10 Video Car ................................................. 71
4.11 Controlled by the APP .......................................... 74
5 Play with Ezblock 83
5.1 Quick Guide on EzBlock ......................................... 83
5.1.1 Servo Adjust .......................................... 83
5.1.2 Install and Configure EzBlock Studio ............................. 85
5.2 Calibrate the Car ............................................. 86
5.3 Move ................................................... 90
5.4 Remote Control ............................................. 94
5.5 Test Ultrasonic Module ......................................... 96
5.6 Test Grayscale Module .......................................... 98
5.7 Color Detection ............................................. 99
5.8 Face Detection .............................................. 102
5.9 Sound Effect ............................................... 105
i

5.10 Background Music ............................................ 107
5.11 Say Hello ................................................. 108
5.12 Music Car ................................................ 111
5.13 Cliff Detection .............................................. 112
5.14 Minecart ................................................. 114
5.15 Minecart Plus ............................................... 117
5.16 Bullfight ................................................. 122
5.17 Beware of Pedestrians .......................................... 124
5.18 Traffic Sign Detection .......................................... 126
5.19 Orienteering ............................................... 130
6 Appendix 137
6.1 PuTTY .................................................. 137
6.2 Install OpenSSH via Powershell ..................................... 139
6.3 About the Battery ............................................ 140
7 FAQ 143
8 Thank You 145
9 Japanese On-line Tutorials 147
10 Copyright Notice 149
ii

SunFounder picar-x
Thanks for choosing our PiCar-X.
The PiCar-X is an AI-driven self-driving robot car for the Raspberry Pi platform, upon which the Raspberry Pi acts
as the control center. The PiCar-X’s 2-axis camera module, ultrasonic module, and line tracking modules can provide
the functions of color/face/traffic-signs detection, automatic obstacle avoidance, automatic line tracking, etc.
PiCar-X has two programming languages: Blockly and Python. No matter what language you program in, you’ll find
detailed steps to teach you everything from configuring the Raspberry Pi to running the relevant example code.
•Play with Python
–This chapter is for those who enjoy programming in Python or want to learn the Python language.
–To get Picar-X working properly, you must install some libraries first.
–The Raspberry Pi configuration and samples code for the PiCar-X are provided in this chapter.
–An APP - SunFounder Controller is also provided to allow you to remotely control the PiCar-X on your
mobile device.
•Play with Ezblock
–In this section, you will use a Blockly based APP, Ezblock Studio, which, like Scratch, allows you to drag
and drop blocks to make Picar-X move.
–It is required to reinstall the SD card with the operating system we provide with pre-installed Ezblock
environment before programming. It is recommended to use a new or unused TF card for this section.
–Ezblock Studio is available for nearly all types of devices, including Macs, PCs, and Androids.
–Ezblock Studio is a good choice if you are 6-12 years old, or don’t have programming skills, or want to
test Picar-X quickly.
About the display language
In addition to English, we are working on other languages for this course. Please contact [e-mail] if
you are interested in helping, and we will give you a free product in return. In the meantime, we recommend using
Google Translate to convert English to the language you want to see.
The steps are as follows.
CONTENTS 1

SunFounder picar-x
• In this course page, right-click and select Translate to xx. If the current language is not what you want, you can
change it later.
• There will be a language popup in the upper right corner. Click on the menu button to choose another language.
• Select the language from the inverted triangle box, and then click Done.
2 CONTENTS

SunFounder picar-x
Content
CONTENTS 3

SunFounder picar-x
4 CONTENTS

CHAPTER
ONE
INTRODUCTION
1.1 The History of Self-driving Cars
Experiments have been conducted on self-driving cars since at least the 1920’s. Promising trials took place in the
1950’s, and work has proceeded forward ever since. The first self-sufficient and truly autonomous cars appeared in the
1980’s, with Carnegie Mellon University’s Navlab and ALV projects in 1984, and Mercedes-Benz and Bundeswehr
University Munich’s Eureka Prometheus Project in 1987. Since the late 1980’s, numerous research organizations
and major automakers have developed working autonomous vehicles, including: Mercedes-Benz, General Motors,
Continental Automotive Systems, Autoliv Inc., Bosch, Nissan, Toyota, Audi, Volvo, Vislab from University of Parma,
Oxford University, and Google. In July 2013, Vislab demonstrated BRAiVE, a vehicle that moved autonomously on
a mixed traffic route open to the public. As of 2019, twenty-nine U.S. states have already passed laws permitting
autonomous cars on public roadways.
Some UNECE members and EU members, including the UK, have enacted rules and regulations related to automated
and fully automated cars. In Europe, cities in Belgium, France, Italy, and the UK have plans in place to operate
transport systems for driverless cars, and Germany, the Netherlands, and Spain have already allowed the testing of
robotic cars in public traffic. In 2020, the UK, the EU, and Japan are already on track to regulate automated cars.
• Reference: History of self-driving cars - Wikipedia
Today, self-driving cars are the closest technological revolution at hand. Some experts predict that by 2025, Level
4 cars are likely to enter the market. The Level 4 cars will allow drivers to divert their attention to something else
entirely, eliminating the need to pay attention to traffic conditions as long as the system is functioning properly.
Level 4 reference:
•SAE Levels of Driving Automation™
•ABI Research Forecasts 8 Million Vehicles to Ship with SAE Level 3, 4 and 5 Autonomous Technology in 2025
5

SunFounder picar-x
Recent rapid advances in software (Artificial Intelligence, Machine Learning), hardware (GPUs, FPGAs, accelerome-
ters, etc.), and cloud computing are driving this technological revolution forward.
• In October 2010, a driverless truck designed by the Italian technology company Vislab took three months to
travel from Italy to China, with a total distance of 8, 077 miles.
• In April 2015, a car designed by Delphi Automotive traveled from San Francisco to New York , traversing
3,400 miles, completing 99 percent of that distance under computer control.
• In December 2018, Alphabet’s Waymo launched a level 4 self-driving taxi service in Arizona , where they had
already been testing driverless cars since 2008. With no one in the driver’s seat, the vehicles operated for more
than a year and traveled over 10 million miles.
• In October 2020, Baidu fully opened its Apollo Robotaxi self-driving cab service in Beijing. The driving routes
cover local residential, commercial, leisure, and industrial parks areas, and offer a fully autonomous driving
system.
However, despite the massive amounts of data collected every day, including training data from real driving records
and simulated scenarios, the complexity of AI models for self-driving cars has not been fully met.
According to RAND’s report , reaching the appropriate level of autonomous learning requires training data from
hundreds of millions, or even hundreds of billions of miles to establish a level of reliability.
So, while the future of self-driving cars is promising and exciting, there are still many more years of development to
go before the technology has matured enough to become fully accessible to the self-driving car market.
The proven way to allow an emerging technology to quickly mature is to make it easily accessible to everyone by
minimizing the market-entry requirements. This is SunFounders motivation for launching PiCar-X.
SunFounders goal is to help beginners, novices, and those who simply just want to learn about autonomous driving, to
understand the development process, the technology, and the latest innovations in self-driving vehicles.
6 Chapter 1. Introduction

SunFounder picar-x
1.2 About PiCar-X
The PiCar-X is an AI-controlled self-driving robot car for the Raspberry Pi platform, upon which the Raspberry Pi acts
as the control center. The PiCar-X’s 2-axis camera module, ultrasonic module, and line tracking modules can provide
the functions of color/face/traffic signs detection, automatic obstacle avoidance, automatic line tracking, etc.
With the SunFounder-designed Robot HAT board, the PiCar-X integrates left/right driving motors, servo motors for
steering and the camera’s pan/tilt functions, and pre-sets the Robot HAT’s ADC, PWM, and Digital I2C pins to
allow for extensions to the standard functionality of the Raspberry Pi. Both a speaker and a bluetooth chip have
been engineered into the Robot HAT for remote control of Text-to-Speech, sound effects, or even background music
functionality.
All of the PiCar-X functions, including GPIO control, computer vision, and deep learning, are implemented through
the open sourced Python programming language, OpenCV’s Computer Vision Library software, and Google’s Tensor-
Flow for deep learning frameworks. Other software has been included to optimize the PiCar-X capabilities, allowing
the user a near-limitless learning environment.
1.2. About PiCar-X 7

SunFounder picar-x
1.3 Deep Learning and Neural Networks
To learn more about deep learning and Neural Networks, SunFounder recommends the following resources:
Machine Learning - Andrew Ng : This course provides a broad introduction to machine learning, datamining, and
statistical pattern recognition.
Neural Networks and Deep Learning : This E-book covers both Neural Networks, a biologically-inspired programming
paradigm that enables a computer to learn from observational data, and Deep learning, a powerful set of techniques
for machine learning in neural networks.
Rethinking the Inception Architecture for Computer Vision : This high-level white-paper explores the methods users
can scale up networks by utilizing added computations as efficiently as possible through factorized convolutions and
aggressive regularization.
8 Chapter 1. Introduction

CHAPTER
TWO
COMPONENT LIST AND ASSEMBLY INSTRUCTIONS
Before assembling the PiCar-X, please first verify that all parts and components have been included. If there are any
missing or damaged components, please contact SunFounder immediately at [e-mail] to resolve the issue
as soon as possible.
Please follow the steps on the following PDF for assembly instructions:
[PDF]Component List and Assembly of PiCar-X.
Note: If the servos has been powered on through the Robot HAT after assembly, do not manually force the steering
gear, as this could cause damage to the servo.
Note:
1. Before assembling, you need to buy 2 18650 batteries and fully charge them, refer to About the Battery.
2. Robot HAT cannot charge the battery, so you need to buy a battery charger at the same time.
9

SunFounder picar-x
10 Chapter 2. Component List and Assembly Instructions

CHAPTER
THREE
ABOUT ROBOT HAT
Left/Right Motor Port
• 2-channel XH2.54 motor ports.
• The left port is connected to GPIO 4 and the right port is connected to GPIO 5.
I2C Pin
11

SunFounder picar-x
• 2-channel I2C pins from Raspberry Pi.
PWM Pin
• 12-channel PWM pins, P0-P12.
ADC Pin
• 4-channel ADC pins, A0-A3.
Digital Pin
• 4-channel digital pins, D0-D3.
Battery Indicator
• Two LEDs light up when the voltage is higher than 7.8V.
• One LED lights up in the 6.7V to 7.8V range.
• Below 6.7V, both LEDs turn off.
USR LED
• Set by your program. (Outputting 1 turns the LED on; Outputting 0 turns it off.)
RST Button
• Short pressing RST Button causes program resetting.
• Long press RST Button till the LED lights up then release, and you will disconnect the Bluetooth.
USR Button
• The functions of USR Button can be set by your programming. (Pressing down leads to a input “0”;
releasing produces a input “1”. )
Power Switch
• Turn on/off the power of the robot HAT.
• When you connect power to the power port, the Raspberry Pi will boot up. However, you will need to
switch the power switch to ON to enable Robot HAT.
Power Port
• 7-12V PH2.0 2pin power input.
• Powering the Raspberry Pi and Robot HAT at the same time.
Note: You can see more details in the Robot HAT Documentation.
12 Chapter 3. About Robot HAT

CHAPTER
FOUR
PLAY WITH PYTHON
For novices and beginners wishing to program in Python, some basic Python programming skills and knowledge of
the Raspberry Pi OS are needed. To start configuring the Raspberry Pi, please reference Quick Guide on Python:
4.1 Quick Guide on Python
This section is to teach you how to install Raspberry Pi OS, configure wifi to Raspberry Pi, remote access to Raspberry
Pi to run the corresponding code.
If you are familiar with Raspberry Pi and can open the command line successfully, then you can skip the first 3 parts
and then complete the last part.
4.1.1 What Do We Need?
Required Components
Raspberry Pi
The Raspberry Pi is a low cost, credit-card sized computer that plugs into a computer monitor or TV, and uses a
standard keyboard and mouse. It is a capable little device that enables people of all ages to explore computing, and to
learn how to program in languages like Scratch and Python.
13

SunFounder picar-x
Power Adapter
To connect to a power socket, the Raspberry Pi has a micro USB port (the same found on many mobile phones). You
will need a power supply which provides at least 2.5 amps.
Micro SD Card
Your Raspberry Pi needs an Micro SD card to store all its files and the Raspberry Pi OS. You will need a micro SD
card with a capacity of at least 8 GB
Optional Components
Screen
To view the desktop environment of Raspberry Pi, you need to use the screen that can be a TV screen or a computer
monitor. If the screen has built-in speakers, the Pi plays sounds via them.
Mouse & Keyboard
When you use a screen , a USB keyboard and a USB mouse are also needed.
HDMI
The Raspberry Pi has a HDMI output port that is compatible with the HDMI ports of most modern TV and computer
monitors. If your screen has only DVI or VGA ports, you will need to use the appropriate conversion line.
Case
You can put the Raspberry Pi in a case; by this means, you can protect your device.
Sound or Earphone
The Raspberry Pi is equipped with an audio port about 3.5 mm that can be used when your screen has no built-in
speakers or when there is no screen operation.
14 Chapter 4. Play with Python

SunFounder picar-x
4.1.2 Installing the OS
Required Components
Any Raspberry Pi 1 * Personal Computer
1 * Micro SD card
Step 1
Raspberry Pi have developed a graphical SD card writing tool that works on Mac OS, Ubuntu 18.04 and Windows,
and is the easiest option for most users as it will download the image and install it automatically to the SD card.
Visit the download page: https://www.raspberrypi.org/software/. Click on the link for the Raspberry Pi Imager that
matches your operating system, when the download finishes, click it to launch the installer.
Step 2
When you launch the installer, your operating system may try to block you from running it. For example, on Windows
I receive the following message:
If this pops up, click on More info and then Run anyway, then follow the instructions to install the Raspberry Pi
Imager.
Step 3
Insert your SD card into the computer or laptop SD card slot.
Step 4
Warning: Upgrading the Raspberry Pi OS to Debian Bullseye will cause some features to not work, so it is
recommended to continue using the Debian Buster version.
In the Raspberry Pi Imager, click CHOOSE OS -> Raspberry Pi OS(other).
4.1. Quick Guide on Python 15

SunFounder picar-x
Scroll down to the end of the newly opened page and you will see Raspberry Pi OS(Legacy) and Raspberry Pi
OS Lite(Legacy), these are security updates for Debian Buster, the difference between them is with or without the
desktop. It is recommended to install Raspberry Pi OS(Legacy), the system with the desktop.
16 Chapter 4. Play with Python