My first build: CAN-Bus and GPS through Bluetooth LE

13

Comments

  • Oh BTW, the ItsyBitsy is awesome!
    This is how small my whole build is now: https://i.imgur.com/1BNYfff.jpg (5x5 mm cells in the background for scale).

    All the wiring is done on the mini breadboard by just a few jumper wires from this kit: https://www.sparkfun.com/products/124
    and is hidden under the ItsyBitsy and the MCP breakout.
  • Figured out a few more things, and here's my first version on GitHub!
    https://github.com/timurrrr/RaceChronoDiyBleDevice

    As you may notice, instead of respecting the "update interval" requested by Arduino, I instead send every Nth message (where N is different for different PIDs). I've observed more stable refresh rates this way, and it makes more sense to me. You might consider changing the RaceChrono DIY BLE protocol to do something like that too.
  • I guess the "update interval" can just be ignored, no protocol changes needed for this. I will refine and fix the device example once I get around to it again, using your experiences and advice posted here!
  • After spending a significant amount of time playing with parameters, writing stress tests, and talking to my more experienced Arduino DIYers, I'm pretty certain that we should power the MCP via regulated 3.3V, not via unregulated USB. Since the voltage on the "USB" pin is not regulated, when nRF is sending data and needs a lot of power, it can cause so much crap on the USB (and thus on VCC of the MCP) that the MCP sends/receives jibberish over SPI or can stop responding altogether.
    On my stress test MCP receives garbage and goes unconscious in just a matter of seconds if VCC is connected to USB; and it works with no issues when VCC is connected to a 3(.3)V pin.

    You might consider updating the wiki in your Github project accordingly.
  • edited September 2020
    I'm powering the whole thing through the USB-pin, not only the MCP board, but also the Adafruit nRF board. So I'm not using the USB-pin as power output, but as an input.

    My build has a separate 12=>5 V regulator/step-down. The regulator built-in to the nRF board is not used for anything.

    As long as the voltage regulator/step-down is good enough I don't think the voltage is going to dip? I'm nowhere near expert on this, but this what I've figured out... My MCP board refuses to work at 3.3V and the input is clearly marked as 5V.
  • Yeah if you're using another regulator when stepping down from 12V to 5V you're probably fine (I'm no expert either) but if someone's getting 5V directly from USB (as your wiki suggests as an option), looks like they should use a regulator.

    I just noticed your wiki says "The 3.3 V outputs are not sufficient, as the board requires 5 V." which contradicts my experience. What's the source for this sentence? The MCP2515 datasheet says it can take as low as 2.7 V.

  • Yeah regulated power is probably a good idea for digital communications circuits. It also depends on max current/power the regulator can support. Generally power supplies should have bypass capacitors to support current spikes due to digital signaling if they don’t exist somewhere on the circuit board. I haven’t taken too close a look at the circuit.
  • edited September 2020
    I’m a little confused by the tutorial. Does racechrono on iOS support the DIY device link over BLE? What’s the bandwidth limit on BLE?

    What’s the pairing process like with the DIY BLE on iOS? Is there any danger that someone can highjack the link or is safe because the DIY interface is transmit only? (I’m concerned with the security of off the shelf BLE OBD interfaces because there doesn’t seem to be any kind of pairing process so I’m considering the DIY route. I’m also concerned that nearby cars in grid can interfere with the connection process making it unreliable.)

    Can the DIY device replace the OBDII interface, reading standard OBD PIDs off the CAN?

    The CAN signals are on the ODB connector right? So I can just tap them off to run into the MCP2515?

    OBD is a polling scheme (request/response) whereas CAN is essentially broadcast it seems? Is it typical that ECUs are just continuously writing their data at some fixed rate to CAN?

    How do you go about reverse engineering the CAN messages? Do you need a logger than can read all the messages at full rate or is the DIY hardware sufficient? If the source is too fast for the link, just pick some random PID at a time and rate limit?
  • > Can the DIY device replace the OBDII interface, reading standard OBD PIDs off the CAN?

    I believe you can make OBD requests via the CAN bus on most modern vehicles.

    > The CAN signals are on the ODB connector right?
    > So I can just tap them off to run into the MCP2515?

    I guess that depends on the vehicle, but for most modern ones yes.
    On some vehicles you might find other ways to get the CAN signals, e.g. on BRZ/86s there's a hidden connector with the CAN pins available:
    https://github.com/timurrrr/RaceChronoDiyBleDevice/blob/master/can_db/ft86.md#connections

    > OBD is a polling scheme (request/response) whereas CAN is essentially broadcast it seems?

    Yup. I believe you can still poll data on the CAN bus (RTR messages?), but I believe it's generally not efficient.

    > Is it typical that ECUs are just continuously writing their data at some fixed rate to CAN?

    I believe so. Might not be true for one-off kinds of messages, e.g. when enabling turn signals.

    > How do you go about reverse engineering the CAN messages?
    > Do you need a logger than can read all the messages at full rate or is the DIY hardware sufficient?
    > If the source is too fast for the link, just pick some random PID at a time and rate limit?

    Those are some good options. You might be able to find some existing info on the internet.

    Other things I've done is creating artificial "Analog 1", "Analog 2", etc. channels in RaceChrono for individual bytes of various PIDs I was interested in (e.g. the most frequent ones*), capturing some of my everyday drives, then cross-referencing the data with the map, and trying to remember what I was doing.
    You'll soon develop a good sense of which bytes provide analog/continuous values and which are bitsets/flags; how to detect signed vs unsigned encoding, and cases when you need to decode a multibyte value instead of separate byte-sized values.

    * — you might find this Arduino sketch useful:
    https://github.com/timurrrr/arduino-CAN/blob/master/examples/PidHistogram/PidHistogram.ino
  • edited September 2020
    Q: Does racechrono on iOS support the DIY device link over BLE?
    Yes.

    Q: What’s the bandwidth limit on BLE?
    It's quite low, but you'll achieve something like 60 messages per second with this setup.

    Q: What’s the pairing process like with the DIY BLE on iOS? Is there any danger that someone can highjack the link or is safe because the DIY interface is transmit only? (I’m concerned with the security of off the shelf BLE OBD interfaces because there doesn’t seem to be any kind of pairing process so I’m considering the DIY route. I’m also concerned that nearby cars in grid can interfere with the connection process making it unreliable.)
    There's no pairing in iOS, and the devices are connected by matching them by name. For OBD-II it may be a security vulnerability of some level, but this DIY device is a "read-only" device, so there's no threat (unless the device can somehow be hacked to do something beyond the API functionality). So only threat is that someone reads your coolant temperature...

    Q: Can the DIY device replace the OBDII interface, reading standard OBD PIDs off the CAN?
    It easily could if you did some more programming. With my code it doesn't.

    Q: The CAN signals are on the ODB connector right? So I can just tap them off to run into the MCP2515?
    OBD-II protocol "runs over" the CAN-Bus.

    Q: OBD is a polling scheme (request/response) whereas CAN is essentially broadcast it seems? Is it typical that ECUs are just continuously writing their data at some fixed rate to CAN?
    As said before, OBD-II protocol "runs over" the CAN-Bus. So CAN-Bus can do all the request/response pairs if needed as well. But usually you're interested in the messages that the ECUs are "broadcasting" with fixed rate.

    Q: How do you go about reverse engineering the CAN messages? Do you need a logger than can read all the messages at full rate or is the DIY hardware sufficient? If the source is too fast for the link, just pick some random PID at a time and rate limit?
    You can start from something as easy as accelerator pedal position. Just manipulate the pedal between fully depressed and not depressed at all. See which byte changes accordingly on each position change. It gets much more complicated for values that cannot be easily manipulated, but you get the idea.


  • > Q: What’s the bandwidth limit on BLE?
    > It's quite low, but you'll achieve something like 60 messages per second with this setup.

    Are you talking about iOS specifically, or BLE in general?
  • edited September 2020
    @timurrrr talking specifically about this DIY build. It achieves pretty much same update rate on both OS. I know you can reach substantial bandwidths with proper hardware and software.
  • edited September 2020
    I see. In my real-world testing I'm consistently getting ~100 messages per second with my build
    https://github.com/timurrrr/RaceChronoDiyBleDevice
  • > I'm pretty certain that we should power the MCP via regulated 3.3V, not via unregulated USB

    While working on another project, I figured out yet another reason for powering the MCP via 3.3V: the nRF uses 3.3V logic, so the external components it talks to should also use 3.3V logic.

    I presume long-term mismatch can cause:
    a) SPI stability issues in the nRF -> MCP direction (as the former sends 3.3V signals, and the latter expects 5V signals)
    b) Damage to the nRF MISO pin (as it gets a 5V signal, but is only designed for 3.3V signals)

    > My MCP board refuses to work at 3.3V

    Curious to find out why! As I've mentioned before, MCP can work with a wide range of voltages.

    Where does it fail: configuration, or transmission?
    Have you tried my patched version of the CAN library?
    https://github.com/timurrrr/arduino-CAN
  • @aol @timurrrr thank you for your very detailed guides and code! I'm having an hard time with the OBDlibk LX so I think I will accelerate my plan of building a DIY probably based on Arduino following your examples.


    I will probably go for a feather instead of itsybitsy because of better availability in Italy (with shipping fees the cost is nearly the same)

    I'm really a n00b so some dumb questions will forllow


    My plan is to use the OBD port and draw 12 V and ground from the right pins, pass them through a 12 V -> 5 V step down and then power the board from there and the MCP2515 from the board. Will this work?

    Can you suggest a good 12v->5v and the best way to connect to the board?

  • edited March 2021
    @captslow
    "How do you go about reverse engineering the CAN messages? Do you need a logger than can read all the messages at full rate or is the DIY hardware sufficient? If the source is too fast for the link, just pick some random PID at a time and rate limit?"

    this is the most difficult part I think. Right now I'm suck with CAN-bus ID 0x090 on my MX5 that I know it is carrying acceleration data but I can't get the grasp on the transfer function.

    Anyway, this is what I did:

    I used a Raspberry Pi 3B that was laying around unused and added a PiCan2 hat. Then I've installed the can-utils tools, amazing piece of CLI software that helps you in visualize and log and manipulate the acquired can data.

    You also need to know the baudrate related to the signal you want to read, usually this information is available on repair shop manuals or the internet :smile:

    I used the cansniffer command, that visualizes with optional color coding the IDs that are changing live:


    Then with paper and pen I started taking notes of what changes when acting on inputs, one by one.

    I started with key in ignition on position but engine not running, I waited for the initial check, and then started moving the steering: oh look, ID 0x081 is changing, bytes 2 and 3.

    I did this for different input, it was easy to spot steering, accelerator pedal percentage, clutch pedal switch, brake pedal switch, etc... Then I started the engine and I got a bunch of other data easily enough, such as rpm. With cansniffer you can also remove ID from whatever is displayed, so once you understand all of an ID you can remove from what you see on screen.

    Once this was done I went out for a couple of a few minutes long test, logging everything with the candump command.

    Then it is just a matter of sitting in front of excel trying to see through the Matrix :smile:

    I started looking for the number of occurrence of each ID. More occurrence in the same amount of time means faster channels, usually where the good stuff is. Slow channel may be related to error lights or other things like water temperature or air temperature.

    Then I went into ploting the signed integer value of each byte of each ID and try to find correlation between what I see and what I know was happening, or to other plot for other IDs.

    Once you know which ID and bytes and length to look at, it is just a matter o finding the right equation to bring the read value to the real world value.

    Also: google is your friend and for sure someone else already tried something for your car, just ask in model specific forums.

    Just know that not everything is running on the CAN bus as a broadcast, but only the information needed for all the different devices to work. I.e. wheels speed and brake pressure are on the CAN bus because are needed for ABS to work and STOP lights to change intensity and body computer to calculate the correct speed, but knock sensor data may not travel in to the network because they are used only for engine management and probably the sensor is directly laced to the engine management computer unit.

    For some information the only way is to interrogate the correct control unit, there are ways of doing this in the CAN-bus protocol.

    Hope this helps :smile:
  • @jeby Nice hacking! Respect! I will need to try those tools too.
  • When you do figure out some useful CAN IDs and conversion functions, please send me a pull request to add them to my GitHub project alongside https://github.com/timurrrr/RaceChronoDiyBleDevice/blob/master/can_db/ft86.md

  • Sure, should I create a mx5nc.md file?

    Would be nice to gather info from various cars

  • Yeah, that would be great! Maybe mazda_mx5_nc.md ?
  • Hi...There are a bunch of trade designs accessible in RaceChrono, counting CSV and RaceRender/DashWare. Any sensibly ancient Android phone with Bluetooth Moo Vitality bolster ought to suffice. You'll need a bigger screen and USB-C quick charging in the event that you need to clearly see the phone screen while driving. If you have got an Android Auto head unit and need to extend the app on the head unit you'll require the phone to be established, and not all phones are simple to root. A few say Motorola E5 could be a great cheap rootable phone for this reason. Otherwise you can purchase a used/refurbished Google Pixel.
  • @aol @timurrrr - thanks so much for your work! Happy to report I’ve got my DIY device up and running and reading codes. Pretty happy with my build (see pic below). I bring CAN H/L, 12V & GND in through an RJ-45 and step the voltage down to 5V which I I feed to the Adafruit BAT power input. (Pic below).

    Sadly, I can’t find any info on CAN IDs for my car, so the next step is code reverse engineering.

    Still have a long way to go, but thrilled with the progress, which I attribute to this forum and the work you have shared.

    Many thanks!


  • @jgunny Great to hear it's working!

    And sorry your car doesn't have much info — that's the worst. Which car is that? My recommendation is start by taking a what I call "PID histogram" (see PidHistogram example in my fork of arduino-CAN) and figure out what are the most frequent CAN IDs. Those CAN IDs are likely to communicate the most useful info the car has. Then try to figure out which ID contains the "accelerator pedal %" data — you can just watch the data in "preview" in the RaceChrono equation editor. In most cars you can see the accelerator pedal data even with the engine off, so that you don't not annoy your neighbors. Then turn the engine on, and try to find the steering wheel and the brake pedal. "RPM" data can be guessed while stationary, "Speed" needs some driving.
    Good luck!
  • Thanks. Car is a 2020 BMW M4 (F82). I have found some CAN ID info for previous generations of the platform (mostly the E46), and plan to try those as a first step (fingers crossed). I'll also try this histogram utility. If I make any progress, I'll create a can_db file. Thanks again.
  • Awesome, thanks! One of my friends has an F82 M4 too, I'm sure he'll appreciate your work!
  • edited January 2022
    Milestone reached. I believe I have deciphered my first CAN ID...steering angle.
    ID: 0x301
    Equation: (3*(bitsToUintLE(raw,16,16)-9984)/128

    Will do some confirmation testing later today and will report back.
  • Are you guys finding the associated equations using google or actually somehow taking the raw data and coming up with the equations. I was looking at @timurrrr github and some of the equations use squareroot etc. seems like it would be hard to get the equation right.

    Very curious about this part. I have PID identification down great on my Motorcycle but want to reverse engineer the equations and that's why i'm interested.
  • edited March 2022
    @Michael86 It's basically comparing to the actual data and doing educated guesses. Some values are just easy to test in a garage, like accelerator position, steering angle, RPM etc. Some are harder like the speed on a car. There you might need to record while driving to be able to figure it out.

    When you decode one PID, you can make assumptions of the other ones. Usually the byte order etc. is the same with all of them. If there's a divider to allow decimal values, it's usually 10^n or 2^n (10, 100, 1000, 16, 128 and 256 are common). If there's negative values, it may be "twos complement" or quite usually the midpoint is the zero (32768 for 16-bit values).
  • @Michael86 The square root I've used is simple geometry / Pythagorean theorem to calculate the combined acceleration knowing the lat/long components.
  • edited March 2022
    Perfect! Thanks @timurrrr and @aol
Sign In or Register to comment.