Module byte_stream

ByteStream Abstractions

When the SDK returns streaming binary data, the inner Http Body is wrapped in ByteStream. ByteStream provides misuse-resistant primitives to make it easier to handle common patterns with streaming data.

Examples

Writing a ByteStream into a file:

use aws_smithy_types::byte_stream::ByteStream;
use std::error::Error;
use tokio::fs::File;
use tokio::io::AsyncWriteExt;
struct SynthesizeSpeechOutput {
    audio_stream: ByteStream,
}

async fn audio_to_file(
    output: SynthesizeSpeechOutput,
) -> Result<(), Box<dyn Error + Send + Sync>> {
    let mut buf = output.audio_stream.collect().await?;
    let mut file = File::open("audio.mp3").await?;
    file.write_all_buf(&mut buf).await?;
    file.flush().await?;
    Ok(())
}

Converting a ByteStream into Bytes

use bytes::Bytes;
use aws_smithy_types::byte_stream::ByteStream;
use std::error::Error;
struct SynthesizeSpeechOutput {
    audio_stream: ByteStream,
}
async fn load_audio(
    output: SynthesizeSpeechOutput,
) -> Result<Bytes, Box<dyn Error + Send + Sync>> {
    Ok(output.audio_stream.collect().await?.into_bytes())
}

Stream a ByteStream into a file

The previous example is recommended in cases where loading the entire file into memory first is desirable. For extremely large files, you may wish to stream the data directly to the file system, chunk by chunk. This is possible using the .next() method.

use bytes::{Buf, Bytes};
use aws_smithy_types::byte_stream::ByteStream;
use std::error::Error;
use tokio::fs::File;
use tokio::io::AsyncWriteExt;
use tokio_stream::StreamExt;
struct SynthesizeSpeechOutput {
    audio_stream: ByteStream,
}

async fn audio_to_file(
    output: SynthesizeSpeechOutput,
) -> Result<(), Box<dyn Error + Send + Sync>> {
    let mut file = File::open("audio.mp3").await?;
    let mut stream = output.audio_stream;
    while let Some(bytes) = stream.next().await {
        let bytes: Bytes = bytes?;
        file.write_all(&bytes).await?;
    }
    file.flush().await?;
    Ok(())
}

Create a ByteStream from a file

Note: This is only available with rt-tokio enabled.

use aws_smithy_types::byte_stream::ByteStream;
use std::path::Path;
struct GetObjectInput {
  body: ByteStream
}

async fn bytestream_from_file() -> GetObjectInput {
    let bytestream = ByteStream::from_path("docs/some-large-file.csv")
        .await
        .expect("valid path");
    GetObjectInput { body: bytestream }
}

If you want more control over how the file is read, such as specifying the size of the buffer used to read the file or the length of the file, use an FsBuilder.

use aws_smithy_types::byte_stream::{ByteStream, Length};
use std::path::Path;
struct GetObjectInput {
    body: ByteStream
}

async fn bytestream_from_file() -> GetObjectInput {
    let bytestream = ByteStream::read_from().path("docs/some-large-file.csv")
        .buffer_size(32_784)
        .length(Length::Exact(123_456))
        .build()
        .await
        .expect("valid path");
    GetObjectInput { body: bytestream }
}

Modules

• error
  Errors related to bytestreams.
• http_body_0_4_x
  This module is named after the http-body version number since we anticipate needing to provide equivalent functionality for 1.x of that crate in the future. The name has a suffix _x to avoid name collision with a third-party http-body-0-4.
• http_body_1_x
  Adapters to use http-body 1.0 bodies with SdkBody & ByteStream

Structs

• AggregatedBytes
  Non-contiguous Binary Data Storage
• ByteStream
  Stream of binary data
• FsBuilder
  Builder for creating ByteStreams from a file/path, with full control over advanced options.

Enums

• Length
  The length (in bytes) to read. Determines whether or not a short read counts as an error.