Student: ThienNgo Nguyen Le
Instructor: Dr. Feng Jiang
Class: Machine Learning - CS3210
Midterm Project-Realistic Driving Video and Data Analysis
1.Detect the accident time from the GPS and sensor data file.
Based on the example code, the G rows and S rows were detected by the index of the row. This method is simple to use but it totally relies on the format of the data. If the data misses some rows then this method won't work. To make it safer when processing the data I detected the G rows and S rows according to the string they start with. In this data G row starts with "[G]" and S row starts with "[S]". This way it can handle the case where some S rows are missing. I also extracted the value of longitudes and latitudes to report them along with the potential crash time instead of just time stamp and x, y, z acceleration values.
I chose to detemine the potential crash time based on the sum of delta x, delta y and delta z acceleration. This will give a more arcuate prediction of the potential crash time. If we predict the potential crash time based on just any one of the variables such as x acceleration, y acceleration, and z acceleration, it should work but in some case it may not very arcuate. For example, if we look at the graph of z acceleration in figure 3 below, there are two movments witht the extreme variation. So if the potential crash time depends on z acceleration, it could be wrong. However, if we look at figure 7, the graph of sum of x, y, and z delta, it clearly show only extreme variation movement (frame 155th, timestape = 15:51:43).
import pandas as pd
import matplotlib.pyplot as plt
input_data = "/Users/thienngole/Desktop/MSU/10-MSU-Spring-2019/CS3120-MachineLearning/Assignment/MidtermProject/000.dat"
start_row = 9 #because the dataset is missing the first G line.
pd.options.display.max_colwidth=100
data = pd.read_csv(input_data, header=None)
col_names = ["frame", "timestamp", "x_accel", "y_accel", "z_accel"
, "longitudes", "latitudes"]
processed_data = pd.DataFrame(index=range(len(data)-start_row) , columns=col_names)
frame = 0;
row_index = 0
for _, row in data.iterrows():
if '[G]' in str(row):
row = str(row)
split_row = row.split("\\t")
timestamp = (split_row[1].split(" ")[1])
longitudes = split_row[2]
latitudes = split_row[3]
frame += 1
else:
processed_data.loc[[row_index], "frame"] = frame
processed_data.loc[[row_index], "timestamp"] = timestamp
processed_data.loc[[row_index], "longitudes"] = longitudes
processed_data.loc[[row_index], "latitudes"] = latitudes
split_row = str(row).split("\\t")
x_accel = split_row[1]
y_accel = split_row[2]
z_accel = split_row[3].split("\n")[0]
processed_data.loc[[row_index], "x_accel"] = float(x_accel)
processed_data.loc[[row_index], "y_accel"] = float(y_accel)
processed_data.loc[[row_index], "z_accel"] = float(z_accel)
row_index += 1
processed_data = processed_data.dropna(how="all")
Calculate delta x, y, and z
processed_data['delta_x'] = (processed_data['x_accel'] - \
processed_data['x_accel'].shift()).abs()
processed_data['delta_y'] = (processed_data['y_accel'] - \
processed_data['y_accel'].shift()).abs()
processed_data['delta_z'] = (processed_data['z_accel'] - \
processed_data['z_accel'].shift()).abs()
Calculate the sum of delta x, y, and y
processed_data['sum_delta'] = (processed_data['delta_x'] + \
processed_data['delta_y'] + \
processed_data['delta_z'])
crash_index = processed_data["sum_delta"].astype(float).idxmax()
crash_frame = processed_data.iloc[crash_index].frame
crash_time = processed_data.iloc[crash_index].timestamp
crash_longitude = processed_data.iloc[crash_index].longitudes
crash_latitude = processed_data.iloc[crash_index].latitudes
crash_x_accel = processed_data.iloc[crash_index].x_accel
crash_y_accel = processed_data.iloc[crash_index].y_accel
crash_z_accel = processed_data.iloc[crash_index].z_accel
print("This would be when the crash happened: \nTime: ",crash_time
, "\nFrame number: ", crash_frame
, "\nLongitude: ", crash_longitude
, "\nLatitude: ", crash_latitude
, "\nX acceleration: ", crash_x_accel
, "\nY acceleration: ", crash_y_accel
, "\nZ acceleration: ", crash_z_accel )
Visualize the potential crash time based on acceleration.
def plot_signal(x, title, y_text):
fig, ax = plt.subplots()
indices = range(len(x))
plt.plot(indices, x, color='C0')
ax.set_xlabel('time, 10 readings per second')
ax.set_ylabel(y_text)
ax.set_title(title)
plt.show()
plot_signal(processed_data.x_accel,'Figure 1: X acceleration', 'acceleration')
plot_signal(processed_data.y_accel,'Figure 2: Y acceleration', 'acceleration')
plot_signal(processed_data.z_accel,'Figure 3: Z acceleration', 'acceleration')
plot_signal(processed_data.delta_x,'Figure 4: X Delta acceleration', 'acceleration')
plot_signal(processed_data.delta_y,'Figure 5: Y Delta acceleration', 'acceleration')
plot_signal(processed_data.delta_z,'Figure 6: Z Delta acceleration', 'acceleration')
plot_signal(processed_data.sum_delta,'Figure 7: Sum Delta acceleration', 'acceleration')
2.Sample one image for each second after the accident time.
3.Perform pedestrian detection on the sample images automatically
import cv2
import math
import matplotlib.pyplot as plt
input_video = ("/Users/thienngole/Desktop/MSU/10-MSU-Spring-2019"
+"/CS3120-MachineLearning/Assignment/MidtermProject/0.MOV")
hog = cv2.HOGDescriptor()
hog.setSVMDetector(cv2.HOGDescriptor_getDefaultPeopleDetector())
cap = cv2.VideoCapture(input_video)
frameRate = cap.get(5) #frame rate 5 is index of "CAP_PROP_FPS"
while(cap.isOpened()):
frame_num = cap.get(1) #current frame number
status, frame = cap.read()
if (status != True):
break
elif (frame_num % math.floor(frameRate) == 0 and frame_num >= 4843.0):
(rects, weights) = hog.detectMultiScale(frame, winStride=(4,4)
, padding=(8,8), scale=1.05)
for (x, y, w, h) in rects:
cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 0, 255), 2)
cv2.imwrite('frame_' + str(int(frame_num)) + '.jpg', frame)
cap.release()
cv2.destroyAllWindows()
print("Done pedestrian detection!")
image_path = ("/Users/thienngole/Desktop/MSU/10-MSU-Spring-2019"
+"/CS3120-MachineLearning/Assignment/MidtermProject")
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
image = mpimg.imread(image_path + "/frame_5916.jpg")
plt.imshow(image)
image = mpimg.imread(image_path + "/frame_5945.jpg")
plt.imshow(image)
image = mpimg.imread(image_path + "/frame_5974.jpg")
plt.imshow(image)
image = mpimg.imread(image_path + "/frame_6003.jpg")
plt.imshow(image)
image = mpimg.imread(image_path + "/frame_6032.jpg")
plt.imshow(image)
image = mpimg.imread(image_path + "/frame_6061.jpg")
plt.imshow(image)
image = mpimg.imread(image_path + "/frame_6090.jpg")
plt.imshow(image)
4.Perform the basic safety analysis for this realistic driving video.
I sample one image per second and the program predicted that the car crash could happen after frame 155 (second 155th) on the video. The video shows that the accident occured at 2:40 (160 second) which is 5 second after the predicted time. This makes sense because the prediction is based on the extreme variation of the acceleration and the ecceleration had an extreme change a few seconds before the crash happen. Then a pedestrian occured at timestape 15:52:21 as how on the first picture above then approach the camera. Overall, the program predict the crash at
Time: 15:51:43
Frame number: 155
Longitude: N39.823961
Latitude: W86.159146
X acceleration: -151.0
Y acceleration: -133.0
Z acceleration: 918.0
which is 5 second earlier than the real crash time at
Time: 15:51:48
Frame number: 160
Longitude: N39.824326
Latitude: W86.159086
X acceleration: -100.0
Y acceleration: 166.0
Z acceleration: 717.0