@PDOT88 not exactly sure. The CAN-bus pins on the OBD-II are not true CAN-bus, they are filtered and still use a request/reply protocol, though you theoretically have access to more data than standard OBD-II with the CAN-bus pins on the OBD-II port. it would be much slower than tapping the CAN-bus wires. i was never able to figure out how to get any real data via OBD-II port via the CAN-bus pins, but once i discovered it was still request/reply protocol, i just didn't bother anymore with it.
i would imagine its very similar to this product, since it uses the OBD-II but seems to offer more track data then standard OBD-II signals
tho they dont ever publish the data rate information, etc, so my guess is it can't be great, given the limited speed of OBD-II port/protocol, and if it was, why wouldn't they advertise it.
at the end of the day, the ESP32 solution is quite good, fast, reliable, but has a steep curve for anyone not really into computer programming and access to soldering tools. @aol_of_RaceChrono did an amazing enabling a DIY api for RaceChrono. this really is a game changer, there just really isn't a lot of room for anyone to productize the DIY devices: low volume, CAN-bus decoding car specific and difficult to figure out, etc.
@PDOT88@dirtyfreebooter There's a similar challenge with 2022+ GR86/BRZ, and one interesting solution is to tap a second OBD-II port into the real CAN bus That allows you to use OBDLink MX+ in "CAN bus" mode, unlocking fast data (... if you know the encoding).
@dirtyfreebooter was looking at your project and was wondering if you have ever seen the RejsaCAN board: https://github.com/MagnusThome/RejsaCAN-ESP32. Looks like your code could be adapted to his design which has some nifty auto off features (but lacks battery support).
@dustinsterk yea, i have seen that, the RejsaCAN. i dont really see why it wouldn't work. having tried a few different ESP32-S3 boards, seems like the only thing that ever needs changing in the code is which GPIO pins are mapped to the CAN RX/TX pins, otherwise things should just work, on an ESP32-S3 board.
i do not know where any good 12v + ground wires are on my car where my powertrain CAN bus connection is (in BMW M3 G8x CAN bus connection is in the passenger footwell), so i just went the battery route. a simple 800mAh LiPo battery lasts literally days, so for my purpose, 6 hours at the track once every other month, i just found it easier then messing with the complex BMW wiring harness without any documentation as to what wire is what.
i really like that the RejsaCAN has the CAN bus piece built in, but without a battery setup, its more hassle to me than the extra feather wing adapter
i am going to try and design and 3d print an enclosure for my setup, Adafruit ESP32-S3 feather + feather wing CAN bus transceiver + LiPo battery something this spring.
@dirtyfreebooter makes sense....that is where I am as well, do I want the battery option or go for 12v+ hardwired with the RejsaCAN. Both options are nice to have.
Please do keep your progress updated related to updates and 3D printed enclosures. I will be following this closely for the spring track season!
I noticed a little time delay in display of the GPS speed and the acceleration (g force) calculated from the GPS data.
In the first screenshot you can see when i roll off the throttle (green line), rpm (violet) has its peak point here and the longitudinal force is zero. Nevertheless, the GPS speed (blue line) continues to increase. https://i.ibb.co/DkRvV0d/Screenshot-20240109-083941-Race-Chrono.jpg
The next screenshot shows the situation 3 measurement points later (25Hz GPS update rate, so 120ms later). GPS speed has reached the peak point here, which seems to make no sense as the longitudinal force indicates that deceleration has already began. https://i.ibb.co/wgbfNfM/Screenshot-20240109-084023-Race-Chrono.jpg
This happens at the end of every straight, I see the longitudinal force already indicates braking while speed has not yet start to decrease.
It is a bit confusing when I try to find my actual braking point from the data.
@GiuseppeBinomi The CAN and GPS data in your build are (probably) coming from two different sources, each having their own processing / data communication buffers and inherent delays. Assuming you have full control of the hardware, you should be able to compensate for that by intentionally buffering + delaying the CAN data by a fixed amount of time. Alternatively, you can try to leverage the "Session Settings" > "CAN-Bus reader delay" / "Auto-detect properties" options.
@timurrrr Thanks for your answer Both GPS and CAN processing is done on the same device. It is not actually CAN but instead measuring raw sensor input (counting rpm sensor pulses, voltage at throttle valve sensor...). Just like you suggested I am able to buffer the CAN data and make a precise synchronization. The PPS signal from GPS module helps to compensate even the delay when reading GPS data. I put all my measured data into the same CAN characteristic (as we can use 16 Byte payload) and send them at the same time with GPS characteristic. So I did everything I could think of to get best possible synchronization.
What I actually wanted to point out is that GPS speed and lateral/longitudinal forces seems not to show up "in line". And that is somehow confusing because forces are calculated from the GPS data... so nothing connected to my additinal CAN data. Maybe anyone can confirm this? It may occur only with fast GPS update rate? I am sure there is a smoothing algorithm in use when calculating the forces, it may be causing a little time shift? Just wild guesses what happens behind.
The next question I had is which of them is displayed with delay, the speed or force? And there I checked with my CAN data which seem to be perfectly synchronized with the calculated forces. Looks like GPS speed is somehow displayed with delay, but I can not proof it yet. When I have the chance to test again on track I will send the GPS speed not only within the GPS characteristic but also as a CAN channel. They should be on the exact same line when displayed in the graph. I can also do little tricks to proof my CAN data is synchronized with GPS data by manipulating the number of satellites. This number is included in the GPS characteristic and it could also be sent as CAN data. So changing this number in a suitable pattern could be used as visual proof for synchronization.
Anyway it's not a big deal and I am just curious about it
"What I actually wanted to point out is that GPS speed and lateral/longitudinal forces seems not to show up "in line". And that is somehow confusing because forces are calculated from the GPS data... so nothing connected to my additinal CAN data. Maybe anyone can confirm this? It may occur only with fast GPS update rate? I am sure there is a smoothing algorithm in use when calculating the forces, it may be causing a little time shift? Just wild guesses what happens behind."
I have observed that the calculated lateral/longitudinal values lag behind the raw lateral/longitudinal values coming directly from the CAN-BUS (this car provides raw acceleration values).
I believe it is expected to have a little delay between the raw data(ex speed) and calculated data (ex lateral force). It depends on how the calculation is done, how many raw values are used around each "time point", the smoothing used etc.
"bleprph" is also included with Espressif esp-idf, and compiles like this on a Windows PC: 1. Use the windows installer from: https://docs.espressif.com/projects/esp-idf/en/stable/esp32/get-started/windows-setup.html 2. Windows->Start "ESP-IDF PowerShell" and write in it: > mkdir $HOME/esp/bleprph > cd $HOME/esp/bleprph > xcopy /e /i $IDF_PATH\examples\bluetooth\nimble\bleprph . > idf.py set-target esp32s3 (or whatever chip/board you use -- and plug it in via USB) > idf.py menuconfig (to change settings, or one can use default settings) > idf.py build > idf.py flash > idf.py monitor 3. Use a phone with a BLE scanner (e.g. "LightBlue") and connect to "nimble-bleprph". The "idf.py monitor" will report the connection and other BLE operations.
Is there anyone in this group that has a good understanding of CAN-bus comms that can help me get up to speed a bit?
I would like to use the obdlink mx+ as CAN-bus transmitter in different cars as I coach multiple people. Can we get anything from CAN databases that Vbox or AIM have built already?
Can anyone explain to me the reverse engineering process to establish PID channels in different cars and how to run the equations for those signals so that the app can read that data?
Will all that data be exported with the .vbo file export so I can analyze all the data in Circuit Tools?
@HONRthedrive - reverse engineering CAN channels is a total PITA. It can be done but requires various tools, a good head for designing software tests, and a ton of patience.
@webkris and I have documented our work on the 718 Cayman platform including all the physical CAN hookups to our RaceChrono pipelines as well as various discussions of tracking down particular CAN signals.
SavvyCAN has a great variety of modes - some visual where it will chart signals of potential interest, and some packet-based if you want to monitor the raw bytes coming in on the sniffer.
That should get you started. As an example of what I'm talking about by "test design" - consider the problem of identifying tire pressure. This requires finding not only 4 different CAN channels (assuming this info is even on a CAN bus you can access) but also de-conflicting which channel goes to which wheel.
I managed to do this for my Cayman with a driveway test like this:
- Connect up CAN sniffer and verify collecting - Start with all 4 wheels same known pressure (e.g. 32 psi) - At precisely 2 minute intervals, remove 5 psi from each tire starting with the front left. - Then at precisely 2 minute intervals, add 10 psi to each tire starting with front left - Finally, at 1 minute intervals remove 5 psi from both fronts, then 1 minute later from both rears. - Disconnect CAN sniffer and verify log file was good
I then diagrammed out what I expect the visual representation of the signal would look like if I came across it in SavvyCAN. By seeing four roughly similarly shaped charts separated by known time intervals I had a clear visual signature I could look for in the charts. And by changing specific wheels at known time differentials I could tell FL from FR from RL from RR. And the 5 psi down then 10 psi up then 5 psi down created a very unique visual signature that was quite distinct from lots of the other CAN bus channel visual "noise" that you will see if you get into this.
I did all this on CAN A on my Cayman (the "Drive" CAN) and came up dry. Then I tapped into a 2nd CAN (the "MMI" CAN) and found exactly the signature I expected. The lesson being that not every data channel is on every CAN bus. In my Cayman there are 5 different CAN buses. I think I had one or two failed runs at the beginning while I was ironing out the test procedure. All in all I think I did four runs in my driveway, each one taking the better part of 30 minutes. I spent at least 4 hours in SavvyCAN going through all the data.
So that one I was at least able to do in my driveway. I also managed to locate actual transmission gear. That required multiple driving tests on the freeway near my house. Again I designed a test layout with specific gear changes at specific times so that there would be a recognizable shape to the chart that I could then search for in Savvy CAN. I think I did 3 freeway runs logging with the sniffer and a couple hours in SavvyCAN.
If you're still game after reading all this, happy to provide coaching.
Comments
PID 0x301
endian: little (intel)
type: unsigned
factor: 0.04395
offset: -1440.11
unit: degrees
start byte: 2
start bit: 0
length (bits): 16 (2 bytes)
range: -1440.11 to 1440.11
rate: 5hz
i will note that PID 0x302 has very similar (not exact) data and comes in at 25hz, but in my testing steering at 5hz is plenty for video syncing.
that PID is coming directly from the appendices of the official m motorsports docs for m235iR
Here are all the pids i have figured out for F/G series:
https://docs.google.com/spreadsheets/d/1O491j8jlEOYR6voFPAYZ2NpbNyCKAgqbxt4HVw0bSDA/edit?usp=sharing
i would imagine its very similar to this product, since it uses the OBD-II but seems to offer more track data then standard OBD-II signals
https://www.getbmwparts.com/oem-parts/bmw-m-performance-drive-analyser-ios-and-android-61432450841
tho they dont ever publish the data rate information, etc, so my guess is it can't be great, given the limited speed of OBD-II port/protocol, and if it was, why wouldn't they advertise it.
at the end of the day, the ESP32 solution is quite good, fast, reliable, but has a steep curve for anyone not really into computer programming and access to soldering tools. @aol_of_RaceChrono did an amazing enabling a DIY api for RaceChrono. this really is a game changer, there just really isn't a lot of room for anyone to productize the DIY devices: low volume, CAN-bus decoding car specific and difficult to figure out, etc.
That allows you to use OBDLink MX+ in "CAN bus" mode, unlocking fast data (... if you know the encoding).
Example product: https://ansixauto.com/2022-brz-gt86-can-adapter/
i do not know where any good 12v + ground wires are on my car where my powertrain CAN bus connection is (in BMW M3 G8x CAN bus connection is in the passenger footwell), so i just went the battery route. a simple 800mAh LiPo battery lasts literally days, so for my purpose, 6 hours at the track once every other month, i just found it easier then messing with the complex BMW wiring harness without any documentation as to what wire is what.
i really like that the RejsaCAN has the CAN bus piece built in, but without a battery setup, its more hassle to me than the extra feather wing adapter
i am going to try and design and 3d print an enclosure for my setup, Adafruit ESP32-S3 feather + feather wing CAN bus transceiver + LiPo battery something this spring.
Please do keep your progress updated related to updates and 3D printed enclosures. I will be following this closely for the spring track season!
In the first screenshot you can see when i roll off the throttle (green line), rpm (violet) has its peak point here and the longitudinal force is zero. Nevertheless, the GPS speed (blue line) continues to increase.
https://i.ibb.co/DkRvV0d/Screenshot-20240109-083941-Race-Chrono.jpg
The next screenshot shows the situation 3 measurement points later (25Hz GPS update rate, so 120ms later). GPS speed has reached the peak point here, which seems to make no sense as the longitudinal force indicates that deceleration has already began.
https://i.ibb.co/wgbfNfM/Screenshot-20240109-084023-Race-Chrono.jpg
This happens at the end of every straight, I see the longitudinal force already indicates braking while speed has not yet start to decrease.
It is a bit confusing when I try to find my actual braking point from the data.
Both GPS and CAN processing is done on the same device. It is not actually CAN but instead measuring raw sensor input (counting rpm sensor pulses, voltage at throttle valve sensor...). Just like you suggested I am able to buffer the CAN data and make a precise synchronization. The PPS signal from GPS module helps to compensate even the delay when reading GPS data. I put all my measured data into the same CAN characteristic (as we can use 16 Byte payload) and send them at the same time with GPS characteristic. So I did everything I could think of to get best possible synchronization.
What I actually wanted to point out is that GPS speed and lateral/longitudinal forces seems not to show up "in line". And that is somehow confusing because forces are calculated from the GPS data... so nothing connected to my additinal CAN data.
Maybe anyone can confirm this? It may occur only with fast GPS update rate?
I am sure there is a smoothing algorithm in use when calculating the forces, it may be causing a little time shift? Just wild guesses what happens behind.
The next question I had is which of them is displayed with delay, the speed or force? And there I checked with my CAN data which seem to be perfectly synchronized with the calculated forces. Looks like GPS speed is somehow displayed with delay, but I can not proof it yet.
When I have the chance to test again on track I will send the GPS speed not only within the GPS characteristic but also as a CAN channel. They should be on the exact same line when displayed in the graph. I can also do little tricks to proof my CAN data is synchronized with GPS data by manipulating the number of satellites. This number is included in the GPS characteristic and it could also be sent as CAN data. So changing this number in a suitable pattern could be used as visual proof for synchronization.
Anyway it's not a big deal and I am just curious about it
Maybe anyone can confirm this? It may occur only with fast GPS update rate?
I am sure there is a smoothing algorithm in use when calculating the forces, it may be causing a little time shift? Just wild guesses what happens behind."
I have observed that the calculated lateral/longitudinal values lag behind the raw lateral/longitudinal values coming directly from the CAN-BUS (this car provides raw acceleration values).
I believe it is expected to have a little delay between the raw data(ex speed) and calculated data (ex lateral force). It depends on how the calculation is done, how many raw values are used around each "time point", the smoothing used etc.
Can the example program "bleprph" be modified to work with RaceChrono?
See also: https://mynewt.apache.org/latest/tutorials/ble/bleprph/bleprph.html
"bleprph" is also included with Espressif esp-idf, and compiles like this on a Windows PC:
1. Use the windows installer from:
https://docs.espressif.com/projects/esp-idf/en/stable/esp32/get-started/windows-setup.html
2. Windows->Start "ESP-IDF PowerShell" and write in it:
> mkdir $HOME/esp/bleprph
> cd $HOME/esp/bleprph
> xcopy /e /i $IDF_PATH\examples\bluetooth\nimble\bleprph .
> idf.py set-target esp32s3 (or whatever chip/board you use -- and plug it in via USB)
> idf.py menuconfig (to change settings, or one can use default settings)
> idf.py build
> idf.py flash
> idf.py monitor
3. Use a phone with a BLE scanner (e.g. "LightBlue") and connect to "nimble-bleprph".
The "idf.py monitor" will report the connection and other BLE operations.
I would like to use the obdlink mx+ as CAN-bus transmitter in different cars as I coach multiple people. Can we get anything from CAN databases that Vbox or AIM have built already?
Can anyone explain to me the reverse engineering process to establish PID channels in different cars and how to run the equations for those signals so that the app can read that data?
Will all that data be exported with the .vbo file export so I can analyze all the data in Circuit Tools?
@webkris and I have documented our work on the 718 Cayman platform including all the physical CAN hookups to our RaceChrono pipelines as well as various discussions of tracking down particular CAN signals.
You can read my series here - https://www.718forum.com/threads/successfully-hacked-my-718-gts-can-bus-for-racechrono-data.31918
You can read Kris' multipart series here - https://planetkris.com/
You'll need a CAN bus sniffer - I use the CL1000 from CSS - https://www.csselectronics.com/products/can-bus-logger-interface-cl1000
If you have a CSS sniffer, you can visualize log files in their free SavvyCAN software - https://www.csselectronics.com/pages/can-bus-interface-savvycan
SavvyCAN has a great variety of modes - some visual where it will chart signals of potential interest, and some packet-based if you want to monitor the raw bytes coming in on the sniffer.
That should get you started. As an example of what I'm talking about by "test design" - consider the problem of identifying tire pressure. This requires finding not only 4 different CAN channels (assuming this info is even on a CAN bus you can access) but also de-conflicting which channel goes to which wheel.
I managed to do this for my Cayman with a driveway test like this:
- Connect up CAN sniffer and verify collecting
- Start with all 4 wheels same known pressure (e.g. 32 psi)
- At precisely 2 minute intervals, remove 5 psi from each tire starting with the front left.
- Then at precisely 2 minute intervals, add 10 psi to each tire starting with front left
- Finally, at 1 minute intervals remove 5 psi from both fronts, then 1 minute later from both rears.
- Disconnect CAN sniffer and verify log file was good
I then diagrammed out what I expect the visual representation of the signal would look like if I came across it in SavvyCAN. By seeing four roughly similarly shaped charts separated by known time intervals I had a clear visual signature I could look for in the charts. And by changing specific wheels at known time differentials I could tell FL from FR from RL from RR. And the 5 psi down then 10 psi up then 5 psi down created a very unique visual signature that was quite distinct from lots of the other CAN bus channel visual "noise" that you will see if you get into this.
I did all this on CAN A on my Cayman (the "Drive" CAN) and came up dry. Then I tapped into a 2nd CAN (the "MMI" CAN) and found exactly the signature I expected. The lesson being that not every data channel is on every CAN bus. In my Cayman there are 5 different CAN buses. I think I had one or two failed runs at the beginning while I was ironing out the test procedure. All in all I think I did four runs in my driveway, each one taking the better part of 30 minutes. I spent at least 4 hours in SavvyCAN going through all the data.
So that one I was at least able to do in my driveway. I also managed to locate actual transmission gear. That required multiple driving tests on the freeway near my house. Again I designed a test layout with specific gear changes at specific times so that there would be a recognizable shape to the chart that I could then search for in Savvy CAN. I think I did 3 freeway runs logging with the sniffer and a couple hours in SavvyCAN.
If you're still game after reading all this, happy to provide coaching.