ESP32/ESP32s3 DIY build progress

making decent headway on the DIY route. i have a few PIDs decoded for G8X m3/m4

50hz RPM
50hz ACCELERATOR PEDAL
50hz BRAKE PRESSURE
25hz GEAR
25hz STEERING ANGLE

RPM and PEDAL come in at 100hz but the Bluetooth LE drivers on all the Arduino devices are terrible. nRF52 or ESP32!

My basic design is an Adafruit Feather ESP32s3 (or Adafruit Huzzah32 ESP32) with Feather Wing CAN bus transceiver (ESP32 has a SJA1000-like CAN controller built into the chip) The ESP32s3 is newer chip with Bluetooth 5.0, but drivers are so terrible, there is not real gains, I just prefer the USB-C vs USB micro B.

I started out with a nRF52 chip, but its performance was maybe 100 frames / second total. The ESP32s3 I can get about 350 frames / second before the Bluetooth stack starts dying.

I have developed my own CAN bus driver, so i can take lots of shortcuts since i only care about reading data. I can get over 2500 frames / second (the max my car broadcasts over the PT-CAN bus) without issue.

The ESP32 is dual-core, so i have CAN bus on one core, Bluetooth on the other, use a FreeRTOS Queue to pass data from the CAN bus interrupt handler to the bluetooth core and send that over Bluetooth LE to racechrono.

RaceChrono screenshot: https://photos.app.goo.gl/Kbn5eceCnLTmVuXQ9
ESP32 build: https://photos.app.goo.gl/yBsnSb2GYug3SNcv7

Next steps are to see if i can write my own BLE slimmed down driver with the ESP32 SDK and get past that 350 frames / second mark.

It would be nice to have a WiFi interface to RaceChrono for DIY devices, as that I feel is much more stable / performance drivers in the arduino world, vs trying to use BLE for performance path. BLE is fine for basic communication, but none of the Arduino drivers are hardened for performance on the BLE side.

Comments

  • edited September 2022
    one other idea i had, since i have so many free CPU cycles on the ESP32 side, was to aggregate PIDs into one BLE message. right now all data is basically 1 or 2 bytes, but the CAN frame is 8 bytes. i could aggregate say all the 50hz messages into one pid, 25hz into another, etc, then use the same PID and different offsets on the RaceChrono iOS/Android side, reducing the BLE messages by potentially 4x, but adding some latency with the aggregation step, but at 25/50/100 hz, it may not matter...

    could also just reduce the BLE payload to PID + value, instead of sending all the extra CAN data so each BLE message would be 4 bytes for PID and 1-2 bytes for data. that would reduce the data payload in each BLE from 12 bytes total to 5-6 bytes total
  • Nice project!
  • edited October 2022
    @dirtyfreebooter:

    email me at magnust@gmail.com if you are interested in trying out my integrated ESP32-S3 CAN board with built in 12V power support from car/bike

    https://github.com/MagnusThome?tab=repositories
  • @MagnusThome sorry, i don't have any 12v available to me easily from where i need to hook into the CANbus and a small 600 mAh battery lasts several days, so plenty of power for a single track day, so i dont really have a good way to use and/or test your setup.
  • πŸ™‚πŸ‘

  • Sorry about late reply. dirtyfreebooter wrote: "It would be nice to have a WiFi interface to RaceChrono for DIY devices, as that I feel is much more stable / performance drivers..."

    A WiFi interface is available, and I have been using it for a year or so -- streaming $RC3 and GPS data via an old ESP8266 serial/WiFi interface at insane speeds. It is definitely faster and more stable than the Bluetooth SPP I used before.
  • @DrMotor is there any API documentation or example github repo for setting up the WiFi connection between device and phone?
  • edited April 2023
    @dirtyfreebooter There's no setup as such. RaceChrono will ask you IP address and TCP port where to connect, when you configure TCP/IP DIY device. You can connect the Wi-Fi from phone's Wi-Fi settings as usual.
  • edited April 2023
    Like AOL said, but more details: You can set up an ESP8266 as WiFi access point so your phone can connect to it. I do it like this on a BlackPill board:

    > send_string( "AT+CWSAP_CUR=\"\",\"\",5,3\r\n" );
    > delay(1000);
    > send_string( "ATE0\r\n" ); // Echo off
    > send_string( "AT+UART_CUR=1000000,8,1,0,0\r\n" ); // 1Mbps, 8 bits, 1 stop bit, no parity, flow control
    > LL_USART_Disable( RC_dev ); // disable to change baud rate
    > LL_USART_SetBaudRate( RC_dev, 96000000U, LL_USART_OVERSAMPLING_16, 1000000 );
    > LL_USART_Enable( RC_dev );
    > send_string( "AT\r\n");
    > send_string( "AT+GMR\r\n");
    > send_string( "AT+CWMODE_CUR=2\r\n"); // 2 = softAP mode (allow phone to connect via WiFi)
    > send_string( "AT+CIPMODE=0\r\n" ); // Prepare normal mode
    > send_string( "AT+CIFSR\r\n" ); // Query IP address
    > send_string( "AT+CIPMUX=1\r\n"); // Enable multiple connections.
    > send_string( "AT+CIPSERVER=1\r\n"); // Create a TCP server. Default port = 333
    > delay(1000); // wait for connection

    The serial port "RC_dev" is connected to the ESP8266 . You might need to insert some delays to let the ESP process the AT-commands -- check via a logic probe! This function sends data to RaceChrono:

    > void send_AT(char* str) {
    > size_t len = strlen(str);
    > if ( len > 0 )
    > {
    > char at_com[]="AT+CIPSEND=0,xxx\r\n";
    > sprintf(&at_com[13],"%03u\r\n",len);
    > send_string(at_com); // Book the transmission and length
    > delay(1); // wait for ">" ToDo: Read and check the response
    > send_string(str);
    > }
    >}

    > void send_string(char* str) {
    > size_t len = strlen(str);
    > for (; len > 0; --len, ++str) {
    > LL_USART_TransmitData8( RC_dev, *str );
    > while (!LL_USART_IsActiveFlag_TXE(RC_dev)) {} // Blocking mode
    > }
    > *(str-len)=0; // clear the buffer, strlen=0
    >}




  • oh but i see you can't add a WiFi DIY device as a CAN bus device. Or at least I was unable to figure that out with v8.0.7 on iOS. BLE with ESP32-S3 on Arduino Release v2.0.10 based on ESP-IDF v4.4.5 .. the BLE stack is certainly the bottleneck in terms of throughput. Being able to send over WiFi would be an interesting test.

    since all CAN bus packets would fit in UDP packet, it would be interesting to test WiFi + UDP, and interesting to pack more than 1 CAN bus data frame into a UDP packet as well, but the dual core ESP32 with CAN bus interrupts on one core and networking on another core seem to do well enough.
  • @dirtyfreebooter, i have been trying to connect my esp32s3 to racechrono, but with no success, bluetooth serial works fine, ble scanner connects,

    But racechrono does not ever attempt to connect, i have tried the diy UUID,

    and the elm327 UUID, but i am stuck, here's my latest attempt,

    #include
    #include
    #include
    #include

    // UUIDs for the BLE service and characteristics of your ELM327 device
    #define SERVICE_UUID "0000fff0-0000-1000-8000-00805f9b34fb"
    #define CHARACTERISTIC_UUID_RX "0000fff1-0000-1000-8000-00805f9b34fb"
    #define CHARACTERISTIC_UUID_TX "0000fff2-0000-1000-8000-00805f9b34fb"

    BLEServer *pServer = NULL;
    BLECharacteristic *pTxCharacteristic;
    bool deviceConnected = false;
    bool oldDeviceConnected = false;

    class ServerCallbacks : public BLEServerCallbacks {
    void onConnect(BLEServer *pServer) {
    deviceConnected = true;
    Serial.println("Device connected");
    };

    void onDisconnect(BLEServer *pServer) {
    deviceConnected = false;
    Serial.println("Device disconnected");
    }
    };

    class CharacteristicCallbacks : public BLECharacteristicCallbacks {
    void onWrite(BLECharacteristic *pCharacteristic) {
    std::string rxValue = pCharacteristic->getValue();

    if (rxValue.length() > 0) {
    Serial.println("Received Value: ");
    for (int i = 0; i < rxValue.length(); i++) {
    Serial.print(rxValue[i]);
    }
    Serial.println();
    }
    }
    };

    void setup() {
    Serial.begin(115200);

    BLEDevice::init("VEEPEAK");
    pServer = BLEDevice::createServer();
    pServer->setCallbacks(new ServerCallbacks());
    BLEService *pService = pServer->createService(SERVICE_UUID);

    BLECharacteristic *pRxCharacteristic = pService->createCharacteristic(
    CHARACTERISTIC_UUID_RX,
    BLECharacteristic::PROPERTY_WRITE | BLECharacteristic::PROPERTY_READ | BLECharacteristic::PROPERTY_NOTIFY
    );
    pRxCharacteristic->setCallbacks(new CharacteristicCallbacks());

    pTxCharacteristic = pService->createCharacteristic(
    CHARACTERISTIC_UUID_TX,
    BLECharacteristic::PROPERTY_READ | BLECharacteristic::PROPERTY_WRITE
    );
    pTxCharacteristic->addDescriptor(new BLE2902());

    pService->start();
    pServer->getAdvertising()->start();
    Serial.println("Waiting for a client connection to notify...");
    }

    void loop() {
    // Handle connection status changes
    if (deviceConnected != oldDeviceConnected) {
    if (!deviceConnected) {
    pServer->startAdvertising();
    }
    oldDeviceConnected = deviceConnected;
    }

    // Read data from Serial and send over BLE
    if (Serial.available()) {
    String data = Serial.readStringUntil('\n'); // Read the incoming data until newline
    if (deviceConnected) {
    pTxCharacteristic->setValue(data.c_str()); // Set the value to the Tx Characteristic
    pTxCharacteristic->notify(); // Notify the connected device
    Serial.print("Transmitted Value: ");
    Serial.println(data); // Print the transmitted data
    }
    }
    }


    Any chance you can tell me whats wrong please, or have an example to get BLE working?

    I am making a Yamaha datalogger for pre 2016 kline,

  • @xt660x honestly i am not 100% sure. ELM327 is for ODB2, not can bus. i don’t really know where you are getting the values for the BLE characteristics, but racechrono specifically is looking for certain UUIDs to discover device.

    // RaceChrono BLE service UUID
    static constexpr uint16_t racechrono_service_uuid = 0x1ff8;

    // RaceChrono uses two BLE characteristics:
    // 0x01 to be notified of data received for those PIDs
    // 0x02 to request which PIDs to send and how frequently
    static constexpr uint16_t can_bus_characteristic_uuid = 0x1;
    static constexpr uint16_t pid_characteristic_uuid = 0x2;

    you can certainly look at my bluetooth setup https://github.com/joeroback/racechrono-canbus/tree/trunk

    but racechrono is specifically looking for 0x1ff8 so the UUIDs you have listed, its certainly never going to be seen by racechrono
  • Thank you for the swift reply, i assumed to clone one of the listed supported BLE odb2 adaptors and racechrono would support it, i also did try the DIY BLE uuids but i was getting nowhere.


    I have taken the BLE section of your code and now racechrono is connected,


    Thanks!
  • edited January 10
    @xt660x

    You can make the ESP32 work as a very basic ELM327 copy with this code
    https://github.com/collin80/ESP32RET
    or for ESP32-S3
    https://github.com/MagnusThome/ESP32S3RET
  • @MagnusThome, Thank you for commenting, I was looking at your code before, i wanted to ask why you removed BLE serial?
  • @MagnusThome i totally agree that a BLE elm327 would be better suited for other apps also, but I could not get any device other than bluetooth serial (android) to connect,

    I assumed it was a UUID issue, but the DIY UUID was the only current way i could ever get working with racechrono.

  • edited January 10

    The S3 does not have Bluetooth Serial which is what ESP32RET is coded for. So to be able to compile for the S3 i made a quick hack removing it. Anyone with some time on their hands can add serial over BLE to the code but I do wonder what phone apps for ELM327 would support that?

  • @MagnusThome Realdash supports BLE odb ELM327,

    I believe torque also supports BLE odb, as does harrys lap timer.
  • IOS only supports BLE as far as i know,
  • edited January 10
    Sorted, I looked back at my veepeak ble+ with ble scanner and I made sure that my code matched the properties correctly, now I have my esp32_s3 using ble connecting to torque, race chrono and real dash, as an odb2 ble adaptor :D

    I will build up on my code and be sure to update my github soon :)


    Thanks to @MagnusThome and @dirtyfreebooter
  • @xt660x

    Your github url?
  • edited January 11


    Here @MagnusThome. I have finished decoding the yds, and I have lots to add to my repo, I am currently just working on the ble side now,

    Uses could be complete dash removal of a pre 2016 yamaha, whilst supporting the yamaha protocol, or alternative dash,


    There's still a lot of people who track pre 2016 r6, and with a simple esp32s3 and an L9637D people are able to produce a cost effective piece of kit.

    Most of the community is locked onto can bus. And provides only support for such.



    But there are many who have contributed to the motorcycle scene, whilst not as common as cars, bikes are still a largely enjoyed sport, bringing the yamaha up to date with an esp32s3 makes sense.

  • edited January 12
    :smiley: Community progress is important for all, I'm glad to see open source support!

    My test board is an UM feather s3.

    But i have a UM nano s3 ready to go when I have finished up.

    My personal plan is to adjust the 5v reg on the dash to support the UM nanos3, tap the RX,TX pads on the dash (which removes the need for the L9637D),

    I can also double up with a decent immo (as the k-line handles immobox/ecu), Yamaha allows ECU flashing via k-line, which would fully disable the bike with a null flash when tampered, added in the fact of open haystack. ( DIY Airtags) You should have a pretty much bulletproof way of getting your bike back if stolen, (Unless the thief thinks to BLE scan)

    https://github.com/seemoo-lab/openhaystack

    Yamaha is Pwned by a simple immobox emulator, which, I won't mention how it's done, is pretty basic, jamming the k-line or null flashing the ecu will solve that issue.

    I also thought of using the s3 devkit as it has USB host, that way a simple USB KKL cable can be converted to function as a data logger, Plenty of options.

    Its winter here in the UK and whilst the weather is too cold to ride for pleasure, i set to study the YDS bus. The AIM kit is too expensive and we all know the possibilities with a simple MCU :)

    415754173-1091266555653188-5638831072759682400-n

    rn-image-picker-lib-temp-7bf972de-afda-4221-9dbd-c51be033f6ea

    End result is to study and reverse my ECU, custom ECUs are always about, but with a datasheet for my ecu, and fully unlocked flashing, its not needed, just time a dedication to the cause :)
  • @MagnusThome @dirtyfreebooter
    https://github.com/terrafirma2021/ESP32-S3-BLE-ELM327

    This is my base, i have lots to do, but soo far torque works without configuration to show speed.

Sign In or Register to comment.