Model Evaluation on Amazon Bedrock allows you to evaluate, compare, and select the best foundation models for your use case. Amazon Bedrock offers a choice of automatic evaluation and human evaluation. You can use automatic evaluation with predefined metrics such as accuracy, robustness, and toxicity. For subjective or custom metrics, such as friendliness, style, and alignment to brand voice, you can set up a human evaluation workflow with a few clicks. Human evaluation workflows can leverage your own employees or an AWS-managed team as reviewers. Model evaluation provides built-in curated datasets or you can bring your own datasets. 

Today, Amazon SageMaker Clarify announces a new capability to support foundation model (FM) evaluations. AWS customers can compare, and select FMs based on metrics such as accuracy, robustness, bias, and toxicity, in minutes.

Today, AWS announces the general availability of Amazon Neptune Analytics, a new analytics database engine. Neptune Analytics makes it faster for data scientists and application developers to get insights and find trends by analyzing graph data with tens of billions of connections in seconds. Neptune Analytics adds to existing Neptune tools and services such as Amazon Neptune Database, Amazon Neptune ML, and visualization tools. Neptune is a fast, reliable, and fully managed graph database service for building and running applications with highly connected datasets, such as knowledge graphs, fraud graphs, identity graphs, and security graphs. With Neptune Analytics, you can find insights in graph data up to 80x faster by analyzing your existing Neptune graph database or graph data from a data lake such as Amazon S3.

Today, Amazon Redshift announces the general availability of support for Apache Iceberg tables. Now, you can easily access your Apache Iceberg tables on your data lake and join it with the data in your data warehouse. This capability offers increased performance whether you are accessing your data lake tables using auto-mounted AWS Glue catalog or external schemas. 

Today, we’re excited to announce the preview of a new smart sifting capability of Amazon SageMaker that automatically inspects and evaluates training data on-the-fly to selectively learn from only the most informative data samples, reducing model training time and cost by up to 35%. You can get started with smart data sifting in minutes without making changes to your existing data pipelines or training scripts.

Amazon OpenSearch Service introduces OR1, the OpenSearch Optimized Instance family, that delivers up to 30% price-performance improvement over existing instances in internal benchmarks and uses Amazon S3 to provide 11 9s of durability. The new OR1 instances are best suited for indexing-heavy workloads, and offers better indexing performance compared to the existing memory optimized instances available on OpenSearch Service. 

We are excited to announce new AWS AI Service Cards, a resource to increase transparency and help customers better understand our AWS AI services, including how to use them in a responsible way. AI service cards are a form of responsible AI documentation that provides customers with a single place to find information on the intended use cases and limitations, responsible AI design choices, and best deployment and operation best practices for our AI Services. They are part of a comprehensive development process we undertake to build our services in a responsible way with fairness, explainability, veracity and robustness, governance, transparency, privacy andsecurity, safety, and controllability.

Today, we are excited to announce the general availability of a simplified developer experience for Amazon SageMaker Pipelines. The improved Python SDK enables you to build Machine Learning (ML) workflows quickly with familiar Python syntax. Key features of the SDK include a new Python decorator (@step) for custom steps, a Notebook Jobs step type, and a workflow scheduler.

You can now use Amazon Bedrock to process prompts in batch to get responses for model evaluation, experimentation, and offline processing. 

Amazon SageMaker Canvas is a no-code tool to build ML models and generate machine learning (ML) predictions. As announced on October 5, customers can access and evaluate foundation models (FMs) from Amazon Bedrock and SageMaker JumpStart to generate and summarize content.