# aws\_s3 Collect logs from **AWS S3** **Type:** `aws_s3` {% code title="sample.yaml" %} ```yaml sources: aws_s3_example: type: aws_s3 sqs.queue_url: https://sqs.us-east-1.amazonaws.com region: bytes auth.strategy: access_key_id auth.access_key_id: '' auth.secret_access_key: '' auth.credentials_file: '' decoding.codec: bytes route_label: '' tls: tls.enabled: false tls.alpn_protocols: '' tls.verify_certificate: true tls.verify_hostname: true tls.ca_file: '' tls.crt_file: '' tls.key_file: '' tls.key_pass: '' # optional: # : ``` {% endcode %} ## Fields Reference Below are tables for **Base**, **TLS**, and **Optional** sections. Columns: Key, Label, Type, Default, Required, Depends on, Enum / Notes. ### Base | Key | Label | Type | Default | Required | Enum / Notes | Depends on | | ------------------------ | ---------------------- | ------ | ----------------------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- | | sqs.queue\_url | SQS Queue URL | string | https://sqs.us-east-1.amazonaws.com | ✅ | The URL of the SQS queue to poll for bucket notifications. | — | | region | Region | enum | bytes | ✅ | The AWS region of the target service. Enum: \["af-south-1", "ap-east-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-south-1", "ap-south-2", "ap-southeast-1", "ap-southeast-2", "ap-southeast-3", "ap-southeast-4", "ap-southeast-5", "ap-southeast-7", "ca-central-1", "ca-west-1", "eu-central-1", "eu-central-2", "eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3", "il-central-1", "me-central-1", "me-south-1", "mx-central-1", "sa-east-1", "us-east-1", "us-east-2", "us-west-1", "us-west-2"] | — | | auth.strategy | Auth Strategy | enum | access\_key\_id | ✅ | The authentication strategy to use. Enum: \["access\_key\_id", "credentials\_file", "imds"] | — | | auth.access\_key\_id | Auth Access Key ID | string | "" | ✅ | The AWS access key ID. | auth.strategy = access\_key\_id | | auth.secret\_access\_key | Auth Secret Access Key | string | "" | ✅ | The AWS secret access key. | auth.strategy = access\_key\_id | | auth.credentials\_file | Auth Credentials File | string | "" | ✅ | Path to the credentials file. | auth.strategy = credentials\_file | | decoding.codec | Decoding Codec | enum | bytes | ✅ | The codec to use for decoding events. Enum: \["avro", "bytes", "gelf", "influxdb", "json", "native", "native\_json", "protobuf", "syslog", "vrl"] | — | | route\_label | Event Routing Label | string | "" | — | The label to use in routing these events to a destination. Defaults to source name-type. | — | ### TLS | Key | Label | Type | Default | Required | Enum / Notes | Depends on | | ----------------------- | -------------------------- | --------------- | ------- | -------- | ------------------------------------------------------------------- | ---------- | | tls.enabled | TLS Enabled | bool | false | — | Whether or not to require TLS for incoming or outgoing connections. | — | | tls.alpn\_protocols | ALPN protocols | string | "" | — | List of supported ALPN protocols. | — | | tls.verify\_certificate | Verify Certificate | bool | true | — | Enable certificate verification. | — | | tls.verify\_hostname | Verify Hostname | bool | true | — | Enable hostname verification. | — | | tls.ca\_file | Certificate Authority File | file\_ca\_file | "" | — | Absolute path to an additional CA certificate file. | — | | tls.crt\_file | Certificate File | file\_crt\_file | "" | — | Absolute path to a certificate file used to identify this server. | — | | tls.key\_file | Private Key | file\_key\_file | "" | — | Absolute path to a private key file used to identify this server. | — | | tls.key\_pass | Private Key Passphrase | string | "" | — | Passphrase used to unlock the encrypted key file. | — | ### Optional | Key | Label | Type | Default | Required | Enum / Notes | Depends on | | --------------------------------------- | ---------------------------------- | --------------- | -------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------- | | description | Description | string | "" | — | The description of the entry. | — | | auth.assume\_role | Auth Assume Role | string | "" | — | The ARN of an IAM role to assume. | — | | auth.external\_id | Auth External ID | string | "" | — | Unique external ID used with assume role. | — | | auth.imds.connect\_timeout\_seconds | Auth IMDS Connect Timeout Seconds | int | 1 | — | Connect timeout for IMDS. | auth.strategy = imds | | auth.imds.max\_attempts | Auth IMDS Max Attempts | int | 4 | — | Number of IMDS retries. | auth.strategy = imds | | auth.load\_timeout\_secs | Auth Load Timeout Secs | int | 30 | — | Timeout for loading any credentials (secs). | auth.strategy = imds | | auth.region | Auth Region | string | "" | — | The AWS region. | — | | auth.session\_name | Auth Session Name | string | "" | — | Role session name for assumed role. | — | | compression | Compression | enum | auto | — | Compression algorithm. Enum: \["auto", "gzip", "none", "zstd"] | — | | decoding.avro.schema | Decoding Avro Schema | string | "" | — | The Avro schema definition. | decoding.codec = avro | | decoding.avro.strip\_schema\_id\_prefix | Strip Avro Schema ID Prefix | bool | true | — | Strip schema ID prefix for Avro datum. | decoding.codec = avro | | decoding.gelf.lossy | Decoding Gelf Lossy | bool | true | — | Replace invalid UTF-8 instead of failing. | decoding.codec = gelf | | decoding.influxdb.lossy | Decoding Influxdb Lossy | bool | true | — | Replace invalid UTF-8 instead of failing. | decoding.codec = influxdb | | decoding.json.lossy | Decoding Json Lossy | bool | true | — | Replace invalid UTF-8 instead of failing. | decoding.codec = json | | decoding.native\_json.lossy | Decoding Native Json Lossy | bool | true | — | Replace invalid UTF-8 instead of failing. | decoding.codec = native\_json | | decoding.protobuf.desc\_file | Decoding Protobuf Desc File | string | "" | — | Path to desc file. | decoding.codec = protobuf | | decoding.protobuf.message\_type | Decoding Protobuf Message Type | string | "" | — | Message type for desc file. | decoding.codec = protobuf | | decoding.syslog.lossy | Decoding Syslog Lossy | bool | true | — | Replace invalid UTF-8 instead of failing. | decoding.codec = syslog | | decoding.vrl.source | Decoding Vrl Source | string | "" | — | VRL program to execute for each event. | decoding.codec = vrl | | decoding.vrl.timezone | Decoding Vrl Timezone | string | local | — | Timezone name for timestamp conversions. | decoding.codec = vrl | | endpoint | Endpoint | string | "" | — | Custom endpoint for AWS-compatible services. | — | | force\_path\_style | Force Path Style | bool | true | — | Use path-style addressing instead of virtual-hosted. | — | | multiline.condition\_pattern | Multiline Condition Pattern | string | "" | — | Regex that determines whether more lines should be read. | — | | multiline.mode | Multiline Mode | enum | continue\_past | — | Aggregation mode; must be used with condition\_pattern. Enum: \["continue\_past", "continue\_through", "halt\_before", "halt\_with"] | — | | multiline.start\_pattern | Multiline Start Pattern | string | "" | — | Regex that matches the start of a new message. | — | | multiline.timeout\_ms | Multiline Timeout MS | int | 600000 | — | Max time to wait for next additional line (ms). | — | | proxy.enabled | Proxy Enabled | bool | false | — | Enable proxying support. | — | | proxy.http | Proxy HTTP | regex | — | — | Proxy endpoint for HTTP traffic. | — | | proxy.https | Proxy HTTPS | regex | — | — | Proxy endpoint for HTTPS traffic. | — | | proxy.no\_proxy | Proxy No Proxy | string | "" | — | Comma-separated list of hosts to avoid proxying. | — | | sqs.client\_concurrency | SQS Client Concurrency | int | — | — | Number of concurrent polling tasks. | — | | sqs.connect\_timeout\_seconds | SQS Connect Timeout | int | — | — | Connection timeout for AWS requests. | — | | sqs.delete\_failed\_message | SQS Delete Failed Message | bool | true | — | Delete non-retryable messages after rejection by sink. | — | | sqs.delete\_message | SQS Delete Message | bool | true | — | Delete the message once processed. | — | | sqs.max\_number\_of\_messages | SQS Max Number of Messages | int | 10 | ✅ | Maximum number of messages to poll in a batch. | — | | sqs.operation\_timeout\_seconds | SQS Operation Timeout | int | — | — | Operation timeout for AWS requests. | — | | sqs.poll\_secs | SQS Poll Secs | int | 15 | ✅ | Seconds to wait while polling for new messages. | — | | sqs.read\_timeout\_seconds | SQS Read Timeout | int | — | — | Read timeout for AWS requests. | — | | sqs.tls\_options.enabled | SQS TLS Enabled | bool | false | — | Require TLS for outgoing connections to SQS. | — | | sqs.tls\_options.alpn\_protocols | SQS TLS ALPN protocols | string | "" | — | List of supported ALPN protocols for SQS client. | — | | sqs.tls\_options.server\_name | SQS TLS Server Name | string | "" | — | Server name for SNI. | — | | sqs.tls\_options.verify\_certificate | SQS TLS Verify Certificate | bool | true | — | Enable certificate verification for SQS. | — | | sqs.tls\_options.verify\_hostname | SQS TLS Verify Hostname | bool | true | — | Enable hostname verification for SQS. | — | | sqs.tls\_options.ca\_file | SQS TLS Certificate Authority File | file\_ca\_file | "" | — | Additional CA certificate file for SQS. | — | | sqs.tls\_options.crt\_file | SQS TLS Certificate File | file\_crt\_file | "" | — | Client certificate file for SQS. | — | | sqs.tls\_options.key\_file | SQS TLS Private Key | file\_key\_file | "" | — | Client private key for SQS. | — | | sqs.tls\_options.key\_pass | SQS TLS Private Key Passphrase | string | "" | — | Passphrase to unlock the SQS key file. | — |