Skip to main content

ADR-008 AWS Bedrock

Status

🤔 Proposed

Context

Our users want to explore and leverage large language models (LLM) to solve business problems.

Our platform lacks the resources required to run these models.

Decision

  • proposed - We will offer Amazon Bedrock to our users. Amazon Bedrock is fully managed large language model platform, which offers many foundation models which be customised privately using techniques such as fine tuning and retrieval-augmented generation (RAG).

Consequences

General consequences

  • Bedrock provides pre-trained models for generations and embeddings
  • Bedrock pricing is based on usage and can vary significantly month-to-month depending on your application’s traffic and costs could spike unexpectedly. Usage is metered and billed per inference request and based on factors like model used, input length, and response length
  • Bedrock models are accessed via an API using AWS permissions

Advantages

  • Serverless access to large language models meaning that our platform and users don’t need to manage and maintain infrastructure
  • Because this is a fully managed service, the compute is managed by AWS and overcomes resourcing limits that currently constrain the platform

Disadvantages

  • Limited model selection, Bedrock offers a few pre-trained models and new models can take time to reach all AWS regions
  • Frankfurt region doesn’t currently offer functionality such as fine-tuning and model training
  • Service is currently only available in Frankfurt and Virginia which raises data sovereignty issues
This page was last reviewed on 17 January 2024. It needs to be reviewed again on 17 July 2024 by the page owner #data-platform-notifications .
This page was set to be reviewed before 17 July 2024 by the page owner #data-platform-notifications. This might mean the content is out of date.