http_client

Collect logs from Http Client

Type: http_client

Sample YAML

{% code title=”sample.yaml” %}

sources:
  http_client_example:
    type: http_client
    endpoint: http://127.0.0.1:9898/logs
    decoding.codec: json
    route_label: ''
    tls:
      tls.enabled: false
      tls.alpn_protocols: ''
      tls.verify_certificate: true
      tls.verify_hostname: true
      tls.ca_file: ''
      tls.crt_file: ''
      tls.key_file: ''
      tls.key_pass: ''
  # optional:
  #   <key>: <value>

{% endcode %}

Fields Reference

Below are tables for Base, TLS, and Optional sections. Columns: Key, Label, Type, Default, Required, Depends on, Enum / Notes.

Base

Key	Label	Type	Default	Required	Enum / Notes	Depends on
endpoint	Endpoint	regex	http://127.0.0.1:9898/logs	✅	The HTTP endpoint to collect events from.	—
decoding.codec	Decoding Codec	enum	json	✅	The codec to use for decoding events. Enum: [“avro”, “bytes”, “gelf”, “influxdb”, “json”, “native”, “native_json”, “protobuf”, “syslog”, “vrl”]	—
route_label	Event Routing Label	string	“”	—	The label to use in routing these events to a destination. Defaults to source name-type.	—

TLS

Key	Label	Type	Default	Required	Enum / Notes	Depends on
tls.enabled	TLS Enabled	bool	false	—	Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See tls.crt_file for more information.	—
tls.alpn_protocols	ALPN protocols	string	“”	—	Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined.	—
tls.verify_certificate	Verify Certificate	bool	true	—	Enables certificate verification. For components that create a server, this requires that the client connections have a valid client certificate. For components that initiate requests, this validates that the upstream has a valid certificate.	—
tls.verify_hostname	Verify Hostname	bool	true	—	Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections.	—
tls.ca_file	Certificate Authority File	file_ca_file	“”	—	Absolute path to an additional CA certificate file.	—
tls.crt_file	Certificate File	file_crt_file	“”	—	Absolute path to a certificate file used to identify this server.	—
tls.key_file	Private Key	file_key_file	“”	—	Absolute path to a private key file used to identify this server.	—
tls.key_pass	Private Key Passphrase	string	“”	—	Passphrase used to unlock the encrypted key file.	—

Optional

Key	Label	Type	Default	Required	Enum / Notes	Depends on
description	Description	string	“”	—	The description of the entry.	—
auth.strategy	Auth Strategy	enum	none	—	The authentication strategy to use. Enum: [“none”, “basic”, “bearer”]	—
auth.token	Auth Token	string	“”	—	The bearer authentication token.	auth.strategy = bearer
auth.user	Auth User	string	“”	—	The basic authentication username.	auth.strategy = basic
auth.password	Auth Password	string	“”	—	The basic authentication password.	auth.strategy = basic
method	Method	enum	get	—	Specifies the method of the HTTP request. Enum: [“delete”, “get”, “head”, “patch”, “post”, “put”]	—
query	Query	string	“”	—	Custom parameters for the HTTP request query string. One or more values for the same parameter key can be provided.	—
proxy.enabled	Proxy Enabled	bool	false	—	Enables proxying support.	—
proxy.http	Proxy HTTP	regex	—	—	Proxy endpoint to use when proxying HTTP traffic.	—
proxy.https	Proxy HTTPS	regex	—	—	Proxy endpoint to use when proxying HTTPS traffic.	—
proxy.no_proxy	Proxy No Proxy	string	“”	—	A list of hosts to avoid proxying.	—
decoding.avro.schema	Decoding Avro Schema	string	“”	—	The Avro schema definition.	decoding.codec = avro
decoding.avro.strip_schema_id_prefix	Strip Avro Schema ID Prefix	bool	true	—	For Avro datum encoded in Kafka messages, the bytes are prefixed with the schema ID.	decoding.codec = avro
decoding.gelf.lossy	Decoding Gelf Lossy	bool	true	—	Determines whether or not to replace invalid UTF-8 sequences instead of failing.	decoding.codec = gelf
decoding.influxdb.lossy	Decoding Influxdb Lossy	bool	true	—	Determines whether or not to replace invalid UTF-8 sequences instead of failing.	decoding.codec = influxdb
decoding.json.lossy	Decoding Json Lossy	bool	true	—	Determines whether or not to replace invalid UTF-8 sequences instead of failing.	decoding.codec = json
decoding.native_json.lossy	Decoding Native Json Lossy	bool	true	—	Determines whether or not to replace invalid UTF-8 sequences instead of failing.	decoding.codec = native_json
decoding.protobuf.desc_file	Decoding Protobuf Desc File	string	“”	—	Path to desc file.	decoding.codec = protobuf
decoding.protobuf.message_type	Decoding Protobuf Message Type	string	“”	—	Path to desc file.	decoding.codec = protobuf
decoding.syslog.lossy	Decoding Syslog Lossy	bool	true	—	Determines whether or not to replace invalid UTF-8 sequences instead of failing.	decoding.codec = syslog
decoding.vrl.source	Decoding Vrl Source	string	“”	—	The Vector Remap Language (VRL) program to execute for each event. Note that the final contents of the . target will be used as the decoding result. Compilation error or use of ‘abort’ in a program will result in a decoding error.	decoding.codec = vrl
decoding.vrl.timezone	Decoding Vrl Timezone	string	local	—	The name of the timezone to apply to timestamp conversions that do not contain an explicit time zone. The time zone name may be any name in the TZ database, or local to indicate system local time.	decoding.codec = vrl
scrape_interval_secs	Scrape Interval Secs	int	15	—	The interval between scrapes. Requests are run concurrently so if a scrape takes longer than the interval a new scrape will be started. This can take extra resources, set the timeout to a value lower than the scrape interval to prevent this from happening.	—
scrape_timeout_secs	Scrape Timeout Secs	int	5	—	The timeout for each scrape request.	—
framing.method	Framing Method	enum	none	—	The framing method. Enum: [“none”, “bytes”, “character_delimited”, “length_delimited”, “newline_delimited”, “octet_counting”]	—
framing.character_delimited.delimiter	Framing Character Delimited Delimiter	string	“”	—	The character that delimits byte sequences.	framing.method = character_delimited
framing.character_delimited.max_length	Framing Character Delimited Max Length	int	—	—	The maximum length of the byte buffer. This length does not include the trailing delimiter.	framing.method = character_delimited
framing.length_delimited.length_field_is_big_endian	Framing Length Delimited Length Field is Big Endian	bool	true	—	Length field byte order (little or big endian).	framing.method = length_delimited
framing.length_delimited.length_field_length	Framing Length Delimited Length Field Length	int	4	—	Number of bytes representing the field length.	framing.method = length_delimited
framing.length_delimited.length_field_offset	Framing Length Delimited Length Field Offset	int	—	—	Number of bytes in the header before the length field.	framing.method = length_delimited
framing.length_delimited.max_frame_length	Framing Length Delimited Frame Length	int	—	—	Maximum frame length.	framing.method = length_delimited
framing.newline_delimited.max_length	Framing Newline Delimited Max Length	int	—	—	The maximum length of the byte buffer.	framing.method = newline_delimited
framing.octet_counting.max_length	Framing Octet Counting Max Length	int	—	—	The maximum length of the byte buffer.	framing.method = octet_counting