Use Python, NimbusML and ML.NET to predict New York taxi fares
There are many popular machine learning libraries for Python. There’s TensorFlow, scikit-learn, Theano, Caffe, and many others.
And in the NET domain we have Microsoft’s new ML.NET machine learning library which can be used in C# and F# applications.
But now Microsoft has created NimbusML, a new library that will let you access the ML.NET machine learning library directly in your Python code!
NimbusML acts as a bridge between the Python process that’s running your app code and the dotNET runtime that’s hosting the ML.NET library. All calls are transparently routed between Python and dotNET.
Naturally I had to try it out. I decided to port my New York taxi price prediction model to NimbusML to see what happens.
I’m always big on writing extremely compact apps and I was happy to get the C# version of the taxi price predictor down to 122 lines of code. And by porting the app to F#, I managed to reduce its size even further to only 69 lines of code.
But how compact will the Python app be?
Let’s find out.
The first thing I’ll need is a data file with transcripts of New York taxi rides. The NYC Taxi & Limousine Commission provides yearly TLC Trip Record Data files which have exactly what I need.
I will download the Yellow Taxi Trip Records from December 2018 and save it as yellow_tripdata_2018–12.csv.
This is a CSV file with 8,173,233 records that looks like this:
There are a lot of columns with interesting information in this data file, but I will only train on the following:
- Column 0: The data provider vendor ID
- Column 3: Number of passengers
- Column 4: Trip distance
- Column 5: The rate code (standard, JFK, Newark, …)
- Column 9: Payment type (credit card, cash, …)
- Column 10: Fare amount