CAN-Bus data logging (was: Custom OBD-II channnels for Miata ND2)

cckcck
edited January 2020 in RaceChrono for Android
Hi. I am a new user. I have just started using RCP with an OBDLink LX in my 2019 Miata. I would like to be able to include current gear and brake position in my video overlays. I did some searching but no luck. I found the following, I don't know if it is relevant.

https://docs.google.com/spreadsheets/d/1uqwIMof9DdaEv0EkWHXe70XNGt0FGeDxpbMAgfKbbdk/edit?usp=sharing

Comments

  • The linked document is not custom OBD-II PIDs, but it is CAN-Bus.

    There is a "beta" feature in RaceChrono that will allow you to log data from CAN-Bus messages, using an OBDLink brand reader. It is a bit hidden as it's not properly tested and documented. But I guess there's no reason not to try it as you already have the hardware and reverse engineered CAN-Bus messages. Also this thread could act as draft documentation for the feature, and when complete will be moved to the support section.

    Background

    CAN-Bus is not OBD-II.

    The CAN-Bus is a communication bus in your car, used for electronic controller units (ECU) communicating with each other. Such as RPM value from engine ECU to the gauge cluster, or accelerator pedal position to engine ECU. Most new vehicles have CAN-Bus, it's been used in cars since early(ish) 2000's, and nowadays in motorbikes too.

    For the CAN-Bus messages to be useful, they can be reverse engineered with right tools (not RaceChrono at the moment), or you might find them already reverse engineered by others. Searching the brand specific internet discussion forums are usually good bet. You will need to know the vehicle specific PID numbers, and the message structures such as byte offset and byte length of the piece of data you're after. Also data scale is helpful when creating the equation.

    OBD-II is a diagnostic standard required by law (Europe and USA), used originally for emissions tests. OBD-II can be implemented on top of CAN-Bus or "K-line Bus" for example. Cars sold in Europe and USA since 2001 implement OBD-II, also even some earlier models that were prepared to be sold after 2001. OBD-II is also implemented in motorbikes that meet the EURO5 emissions standard (some other bikes too, but it's kind of rare).

    Setup for CAN-Bus logging using OBDLink reader

    1) Enable "RaceChrono > Settings > Expert settings > Experimental devices"
    2) Add "OBDLink LX/MX/MX+ Bluetooth (CAN-Bus)" from "RaceChrono > Settings > Add other device". Notice do not add it from "Add OBD-II reader", as it will just add an OBD-II reader that will not be used for CAN-Bus.
    3) Now a hidden feature is revealed, you can add CAN-Bus channels in "RaceChrono > Settings > Vehicle profile" screen.

    Adding CAN-Bus channels

    Go to "RaceChrono > Settings > Vehicle profile" and add new CAN-Bus channel. Then fill the fields and save the channel. Then you're ready to test it!

    For example the speed channel from your spreadsheet:
    1) Channel: Speed
    2) PID: 514 (or 0x202)
    3) Equation: In the spreadsheet it says scale 1/160 for MPH. So equation "bytesToUint(raw, 2, 2) / 160.0" would get you MPH. But RaceChrono needs meters per second (m/s), and the original value seems to be (kph with 1/100 scale), so working equation will be "bytesToUint(raw, 2, 2) / 360.0".

    Tutorial on creating these equations: https://racechrono.com/support/equations
  • Thanks. I will give this a try soon.
  • Hi. I finally started to look at this. So far I am getting some data, I'm working on gear. The info in the spreadsheet seems correct, raw values follow the described scheme.

    However it leads to a pretty good test of my binary logic to express it as a formula to map the raw values to actual gear. I think I have it, but maybe there is a bug in rcp parsing, it says 'brackets mismatch' but I don't see how. Here is my gear formula:

    ((G&16)>>4)*1+((G&8)>>3)*2+((((G&16)>>4)^1)&0x1)*((((G&8)>>3)^1)&1)*min(3,max(6,(10-(G&7))))
  • @cck yeah it's a bug in balancing the parentheses. For now use something less complex, I will fix this for the next major version :)
  • I think I found a variation for gear that doesn't trigger the bug.

    On a separate note, brake pressure isn't working at all. Maybe the PID has changed, my car is a 2019 and the spreadsheet is pre-2019 I believe.

  • Is this limited to OBD Link devices? I have OBDKey bluetooth dongle that should be capable communicating via CAN but I never figured out what software is able to use it to full extent...
    http://www.obdkey.com/vehiclediagnostics.asp#VehicleInfo
    I basically only used it as generic OBD dongle...
  • aolaol
    edited February 2020
    @MAV Yes, limited to OBDLink for now.
  • edited July 2020
    Wow, I wish I found this thread before I've purchased the components for a DIY CAN bus reader. :) I just tried reading some CAN data my BRZ using OBDLink MX+, and it did read some values, most notably the brake pressure.

    However, it is unable to keep refreshing the values.
    When I enable "Test connection" in "Vehicle profile", the "CAN-Bus channels" show the preview of the values once,, but don't update afterwards. "Use live data" doesn't become available in the "Channel editor". Similarly, when I start a session, the values don't get updated once they are shown.
    Same results whether I try multiple different CAN PIDs or just one CAN PID.

    Should I grab any logs to help you debug the issue?

    Also, is it correct to assume that using the same OBDLink for reading via OBD-II and CAN bus at the same time will never be possible? I've tried that and it appears that the app keeps connecting to one protocol while disconnecting from the other, then switches the protocols and the process repeats.
  • aolaol
    edited July 2020
    @timurrrr Does it work fine when recording a session? So only a problem with the "Test connection"?

    The current command set does not easily allow OBD-II polling and CAN-Bus monitoring at the same time. On a DIY device it would be easily possible. Or with two readers.
  • > The current command set does not easily allow OBD-II polling and
    > CAN-Bus monitoring at the same time.
    Understood

    > Does it work fine when recording a session? So only a problem with the "Test connection"?
    >> Similarly, when I start a session, the values don't get updated once they are shown.
  • edited September 2020
    @cck i wonder whether you could find the information of brake pressure
  • @cck @ppanda I found how to get data about the brake pedal via OBD!

    Turns out that some of the Ford PIDs still work on Mazdas as well.
    I've tested on my gf's ND2, but I presume they will work on ND1 also.

    Brake position (%)
    OBD-II header: 0x760 (not sure if necessary, what's the default value?)
    PID: 0x222b0d
    Equation: max(0, bytesToInt(raw, 1, 2)) * 0.67
    (You can try other multipliers if you find the number to be too high or too low for you. You can also wrap it into min(100, ...) to limit to 100%.)

    Accelerator position (%)
    PID: 0x22032b
    Equation: B/2
    (Surprisingly, the standard OBD PIDs for accelerator communicate values in some weird ranges. This PID gives data in a 0–200 range, but at least that looks deliberate)

    These two, along with the standard OBD PIDs for Speed and RPM, should be a very good set of "Fast channels" to use with RaceChrono. Given the limited bandwidth of the OBD protocol, I wouldn't bother adding any other PIDs as "fast" for track/autox logging.

    I also found how to read tire pressure, if your Miata has TPMS sensors (I believe earlier model year NDs had indirect TPMS only):
    OBD-II header: 0x720
    PID: 0x222a05 FL, 0x222a07 FR, 0x222a06 RL, 0x222a08 RR
    Equation: B * 1.373
    You'll probably want to use these PIDs as "Slow channels", as they don't change that often, and would interfere too much with the more time sensitive data if you make them "Fast channels".

    @aol Consider renaming the thread back to "OBD PIDs for ND2 Miatas" :)
  • I did not make progress beyond current gear. Kind of lucky I just checked this thread, super excited to try this new stuff! Thanks @timurrrr !

  • edited September 2020
    @timurrrr
    Thank you for your work!!!! I'll try it right away and update my result soon :smiley:

    *update
    It works perfectly!! Thank you once again. You are my hero!
  • Does anyone know of a software that can turn obdlink into a CAN logger on a range of PIDs? Or do I need to hack my own/get a can logger?

  • @captslow no but if you're in to hacking, you could monitor the CAN-Bus with simple ATMA command, and then do filtering on the output.
  • @cck which PID you use for raw data to calculate gear in ND2?
  • Hey @timurrrr how are you getting the brake pressure from OBDLink MX+ in CAN mode? I just tested on my BRZ and failed to get any reading from lower PIDs (0x18, 0xD0~D4). Test connection just showed zeros. But I did get fast updates from 0x140 and up (btw HUGE thanks for your DIY GitHub repo for the info :+1: ).
  • I figured it out. It was not that it failed to get any reading. It was exactly the same issue @timurrrr had above. The value only gets updated once when the "Test connection" starts and never changes. If I push the brake pedal down and start the connection I'll get a correct reading. After that, it won't change at all. The same for actually recording a session. @aol is this a problem with the OBDLink MX+ hardware? Or something that can be fixed on the software side? (I hope). And how can we help to collect some logs for triage?
  • Well, I sacrificed my weekend sleep and I think I'm onto something now. I took a raw device_output.obd file from a test session and found these lines:

    <--01:25:16.442 OUT-->STFCP\r\r
    <--01:25:16.485 IN -->OK\r\r>
    <--01:25:16.485 OUT-->STFAP 360,7FF\r
    <--01:25:16.522 IN -->OK\r\r>
    <--01:25:16.522 OUT-->STFAP 140,7FF\r
    <--01:25:16.592 IN -->OK\r\r>
    <--01:25:16.593 OUT-->STFAP 140,7FF\r
    <--01:25:16.628 IN -->OK\r\r>
    <--01:25:16.629 OUT-->STFAP 360,7FF\r
    <--01:25:16.734 IN -->OK\r\r>
    <--01:25:16.734 OUT-->STFAP D1,7FF\r
    <--01:25:16.772 IN -->?\r\r>
    <--01:25:16.772 OUT-->STFAP D1,7FF\r
    <--01:25:16.808 IN -->?\r\r>
    <--01:25:16.809 OUT-->STFAP D1,7FF\r
    <--01:25:16.877 IN -->?\r\r>
    <--01:25:16.877 OUT-->STFAP 140,7FF\r
    <--01:25:16.920 IN -->OK\r\r>
    <--01:25:16.920 OUT-->STFAP 361,7FF\r
    <--01:25:16.985 IN -->OK\r\r>

    I believe here RaceChrono is setting pass filters for the defined channels. Here you can see the two 0x360 PID I set for oil temp and coolant temp, three 0x140 filters for accelerator position, engine rpm, and clutch position, one 0x361 filter for gear. These filters were set correctly as OBDLink replied OK.

    The three lines trying to set the D1 filter (which is for brake position) are interesting. OBDLink replied "?" so they all failed. And I only have one channel that requests 0xD1 so I guess RaceChrono tried three times before aborting?

    Then I found the OBDLink® Family Reference and Programming Manual (FRPM) and the documentation for the STFAP (which is deprecated by STFPA) says:

    STFPA [pattern], [mask]
    Add a pass filter. Takes two parameters: pattern
    and mask. Pattern and mask can be any length from 0
    to 5 bytes (0 to 10 ASCII characters), ***but both have to
    be the same length.*** The messages are matched MSB
    first, up to the filter length. Messages shorter than the
    filter length, will not match that filter.
    If an odd number of ASCII characters is specified, a
    leading 0 will be added to the first byte.
    (^but no leading zero will be added for an even number! D1 does not automatically become 0D1 and then 00D1.)

    So here we go, that's the problem. The reason why a lower PID cannot be correctly recognized is that, well, it's lower and its most-significant byte is zero, and RaceChrono does not properly add the leading zero for the STFAP command. I think sending "STFAP 0D1,7FF\r" would correctly set the pass filter.

    And the reason why the brake position will be set once (well actually twice) when the connection starts is that RaceChrono also sends two STM commands (one STM and one carriage return which repeats the first STM) which monitors the OBS bus with no filters right after initializing the connection.

    @aol could you please confirm the above hypothesis? Also, I haven't tested cases for 29-bit CAN vehicles which have 4 bytes for CAN ID so not sure if there's a similar problem with that.
  • Fixed in upcoming v7.2.0.
  • Wow, awesome discovery! So chances are I won't need the DIY device anymore? 🤔

Sign In or Register to comment.