Ingesting events

Sending data to Better Stack Warehouse is as easy as sending an HTTP request.

This endpoint allows you to ingest a single event or a list of events. The events can be encoded in JSON or preferably in a more efficient MessagePack.

Start by creating a Warehouse source in the data region of your choice.

Unauthorized

Couldn't parse JSON content.

payload reached size limit

Examples

Single event

Send a single event using cURL:

curl -X POST https://$INGESTING_HOST \
     -H "Authorization: Bearer $SOURCE_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{"message":"My first event","nested":{"values":123}}'

curl -X POST https://$INGESTING_HOST \
     -H "Authorization: Bearer $SOURCE_TOKEN" \
     -H "Content-Type: application/x-ndjson" \
     -d '{"message":"My first event","nested":{"values":123}}'

# Python is required to prepare the binary data in this example
python3 -c 'import msgpack; \
     print(msgpack.packb( \
          {"message":"My first event","nested":{"values":123}} \
     ).hex())' \
     | xxd -r -p \
     | curl -X POST https://$INGESTING_HOST \
          -H "Authorization: Bearer $SOURCE_TOKEN" \
          -H "Content-Type: application/msgpack" \
          --data-binary @-

fetch('https://$INGESTING_HOST', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer $SOURCE_TOKEN',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify(
    {"message": "My first event", "nested": {"values": 123}}
  ),
})
.then(res => { if (!res.ok) console.error('Err:', res.status) })
.catch(error => console.error('Err:', error.message));

Multiple events

Send multiple events using cURL:

curl -X POST https://$INGESTING_HOST \
     -H "Authorization: Bearer $SOURCE_TOKEN" \
     -H "Content-Type: application/json" \
     -d '[{"message":"A"},{"message":"B"}]'

curl -X POST https://$INGESTING_HOST \
     -H "Authorization: Bearer $SOURCE_TOKEN" \
     -H "Content-Type: application/x-ndjson" \
     -d $'{"message":"A"}\n{"message":"B"}'

# Python is required to prepare the binary data
python3 -c 'import msgpack; \
     print(msgpack.packb( \
          [{"message":"A"},{"message":"B"}] \
     ).hex())' \
     | xxd -r -p \
     | curl -X POST https://$INGESTING_HOST \
          -H "Authorization: Bearer $SOURCE_TOKEN" \
          -H "Content-Type: application/msgpack" \
          --data-binary @-

fetch('https://$INGESTING_HOST', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer $SOURCE_TOKEN',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify(
    [{"message": "A"}, {"message": "B"}]
  ),
})
.then(res => { if (!res.ok) console.error('Err:', res.status) })
.catch(error => console.error('Err:', error.message));

Sending timestamps

By default, the time of the event will be the time of receiving it. You can override this by including a field dt containing the event time either as:

• UNIX time in whole seconds, milliseconds, or nanoseconds. 1672490759, 1672490759123, 1672490759123456000
• String formatted according to RFC 3339. 2022-12-31T13:45:59.123456Z, 2022-12-31 13:45:59.123456+02:00

Alternatively, you can use ISO 8601, as it will most likely use a format compatible with RFC 3339. In MessagePack, you can also use the timestamp extension type.

In case the timestamp can't be parsed, we save it as a string, but revert to using the reception time as the event time.

curl -X POST https://$INGESTING_HOST \
     -H "Authorization: Bearer $SOURCE_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{"message":"I have arrived on time","dt":"2023-08-09 07:03:30+00:00"}'

Request limits

The maximum allowed size of a single request that might contain many events is 10 MiB of compressed data.

Each event is limited to 1 MiB maximum size, but we recommend keeping a single event under 100 KiB for the best experience.

There is no limit to the number of requests you can send.