Starting today, Amazon Q Developer can document your code by automatically generating readme files and data-flow diagrams within your projects. 

Today, developers report they spend an average of just one hour per day coding. They spend most of their time on tedious, undifferentiated tasks such as learning codebases, writing and reviewing documentation, testing, managing deployments, troubleshooting issues or finding and fixing vulnerabilities. Q Developer is a generative AI-powered assistant for designing, building, testing, deploying, and maintaining software. Its agents for software development have a deep understanding of your entire code repos, so they can accelerate many tasks beyond coding. With this new capability, Q Developer can help you understand your existing code bases faster, or quickly document new features, so you can focus on shipping features for your customers.

This capability is available in the integrated development environment (IDE) through a new chat command: /doc . You can get started generating documentation within the Visual Studio Code and IntelliJ IDEA IDEs with an Amazon Q Developer Free Tier or Pro Tier subscription. For more details on pricing, see Amazon Q Developer pricing .

This capability is available in all AWS Regions where Amazon Q Developer is available . To get started with generating documentation, visit Amazon Q Developer or read the news blog . 

With Amazon Bedrock Model Distillation, customers can use smaller, faster, more cost-effective models that deliver use-case specific accuracy that is comparable to the most capable models in Amazon Bedrock.

Today, fine-tuning a smaller cost-efficient model to increase its accuracy for a customers’ use-case is an iterative process where customers need to write prompts and response, refine the training dataset, ensure that the training dataset captures diverse examples, and adjust the training parameters.

Amazon Bedrock Model Distillation automates the process needed to generate synthetic data from the teacher model, trains and evaluates the student model, and then hosts the final distilled model for inference. To remove some of the burden of iteration, Model Distillation may choose to apply different data synthesis methods that are best suited for your use-case to create a distilled model that approximately matches the advanced model for the specific use-case. For example, Bedrock may expand the training dataset by generating similar prompts or generate high-quality synthetic responses using customer provided prompt-response pairs as golden examples.

Learn more in our documentation and blog .
 

Today, AWS announces the next generation of Amazon SageMaker, including the preview launch of Amazon SageMaker Unified Studio, an integrated data and AI development environment that enables collaboration and helps teams build data products faster. SageMaker Unified Studio brings together familiar tools from AWS analytics and AI/ML services for data processing, SQL analytics, machine learning model development, and generative AI application development. Amazon SageMaker Lakehouse, which is accessible through SageMaker Unified Studio, provides open source compatibility and access to data stored across Amazon Simple Storage Service (Amazon S3) data lakes, Amazon Redshift data warehouses, and third- party and federated data sources. Enhanced governance features are built in to help you meet enterprise security requirements.

SageMaker Unified Studio allows you to find, access, and query data and AI assets across your organization, then work together in projects to securely build and share analytics and AI artifacts, including data, models, and generative AI applications. SageMaker Unified Studio offers the capabilities to build integrated data pipelines with visual extract, transform, and load (ETL), develop ML models, and create custom generative AI applications. New unified Jupyter Notebooks enable seamless work across different compute resources and clusters, while an integrated SQL editor lets you query your data stored in various sources—all within a single, collaborative environment. Amazon Bedrock IDE, formerly Amazon Bedrock Studio, is now part of the SageMaker Unified Studio in public preview, offering the capabilities to rapidly build and customize generative AI applications. Amazon Q Developer, the most capable generative AI assistant for software development, is integrated into SageMaker Unified Studio to accelerate and streamline tasks across the development lifecycle.

For more information on AWS Regions where SageMaker Unified Studio is available in preview, see Supported Regions .

To get started, see the following resources:

• SageMaker overview
• SageMaker documentation
• SageMaker in the AWS Management Console  

Today, we are excited to announce the general availability of AWS Glue 5.0. With AWS Glue 5.0, you get improved performance, enhanced security, support for Amazon Sagemaker Unified Studio and Sagemaker Lakehouse, and more. AWS Glue 5.0 enables you to develop, run, and scale your data integration workloads and get insights faster.

AWS Glue is a serverless, scalable data integration service that makes it simple to discover, prepare, move, and integrate data from multiple sources. AWS Glue 5.0 upgrades the engines to Apache Spark 3.5.2, Python 3.11, and Java 17, with new performance and security improvements. Glue 5.0 updates open table format support to Apache Hudi 0.15.0, Apache Iceberg 1.6.1, and Delta Lake 3.2.0 so you can solve advanced use cases around performance, cost, governance, and privacy in your data lakes. AWS Glue 5.0 adds Spark native fine grained access control with AWS Lake Formation so you can apply table, column, row, and cell level permissions on Amazon S3 data lakes. Finally, Glue 5.0 adds support for Sagemaker Lakehouse to unify all your data across Amazon S3 data lakes and Amazon Redshift data warehouses.

AWS Glue 5.0 is generally available today in US East (N. Virginia), US East (Ohio), US West (Oregon), Europe (Ireland), Europe (London), Europe (Stockholm), Europe (Frankfurt), Asia Pacific (Hong Kong), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Canada (Central), and South America (São Paulo) regions.

To learn more, visit the AWS Glue product page and documentation .

Now generally available, Amazon Q in QuickSight provides users with unified insights from structured and unstructured data sources through integration with Amazon Q Business. While structured data is managed in conventional systems, unstructured data such as document libraries, webpages, images and more has remained largely untapped due to its diverse and distributed nature.

With Amazon Q in QuickSight business users can now augment insights from traditional BI data sources such as databases, data lakes and data warehouses, with contextual information from unstructured sources. Users can get augmented insights within QuickSight’s BI interface across multi-visual Q&A and Data Stories. Users can use multi-visual Q&A to ask questions in natural language and get visualizations and data summaries augmented with contextual insights from Amazon Q Business. With data stories in Amazon Q in QuickSight users can upload documents, or connect to unstructured data sources from Amazon Q Business to create richer narratives or presentations explaining their data with additional context. This integration enables organizations to harness insights from all their data without the need for manual collation, leading to more informed decision-making, time savings, and a significant competitive edge in the data-driven business landscape.

This new capability is generally available to all Amazon QuickSight Pro Users in US East (N. Virginia), and US West (Oregon) AWS Regions .

To learn more visit the AWS Business Intelligence Blog, the Amazon Q Business What’s New Post and try QuickSight free for 30-days .
 

Amazon SageMaker Lakehouse announces unified data connectivity capabilities to streamline the creation, management, and usage of connections to data sources across databases, data lakes and enterprise applications. SageMaker Lakehouse unified data connectivity provides a connection configuration template, support for standard authentication methods like basic authentication and OAuth 2.0, connection testing, metadata retrieval, and data preview. Customers can create SageMaker Lakehouse connections through SageMaker Unified Studio (preview), AWS Glue console, or custom-built application using APIs under AWS Glue .

With SageMaker Lakehouse unified data connectivity, a data connection is configured once and can be reused by SageMaker Unified Studio, AWS Glue and Amazon Athena for use cases in data integration, data analytics and data science. You will gain confidence in the established connection by validating credentials with connection testing. With the ability to browse metadata, you can understand the structure and schema of the data source and identify relevant tables and fields. Lastly, the data preview capability supports mapping source fields to target schemas, identifying needed data transformation, and receiving immediate feedback on the source data queries.

SageMaker Lakehouse unified connectivity is available where Amazon SageMaker Lakehouse or AWS Glue is available. To get started, visit AWS Glue connection documentation or the Amazon SageMaker Lakehouse data connection documentation.

Today, AWS announces the general availability of Amazon Elastic Compute Cloud (Amazon EC2) Trn2 instances and preview of Trn2 UltraServers, powered by AWS Trainium2 chips. Available via EC2 Capacity Blocks, Trn2 instances and UltraServers are the most powerful EC2 compute solutions for deep learning and generative AI training and inference.

You can use Trn2 instances to train and deploy the most demanding foundation models including large language models (LLMs), multi-modal models, diffusion transformers and more to build a broad set of AI applications. To reduce training times and deliver breakthrough response times (per-token-latency) for the most capable, state-of-the-art models you might need more compute and memory than a single instance can deliver. Trn2 UltraServers is a completely new EC2 offering that uses NeuronLink, a high-bandwidth, low-latency fabric, to connect 64 Trainium2 chips across 4 Trn2 instances into one node unlocking unparalleled performance. For inference, UltraServers help deliver industry-leading response times to create the best real-time experiences. For training, UltraServers boost model training speed and efficiency with faster collective communication for model parallelism as compared to standalone instances.

Trn2 instances feature 16 Trainium2 chips to deliver up to 20.8 petaflops of FP8 compute, 1.5 TB high bandwidth memory with 46 TB/s of memory bandwidth, and 3.2 Tbps of EFA networking. Trn2 UltraServers feature 64 Trainium2 chips to deliver up to 83.2 petaflops of FP8 compute, 6 TB of total high bandwidth memory with 185 TB/s of total memory bandwidth, and 12.8 Tbps of EFA networking. They both are deployed in EC2 UltraClusters to provide non-blocking, petabit scale-out capabilities for distributed training. Trn2 instances are generally available in the trn2.48xlarge size in the US East (Ohio) AWS Region through EC2 Capacity Blocks for ML .

To learn more about Trn2 instances and request access to Trn2 UltraServers please visit the Trn2 instances page . 

Amazon SageMaker now supports connectivity, discovery, querying, and enforcing fine-grained data access controls on federated sources when querying data with Amazon Athena. Athena is a query service that makes it simple to analyze your data lake and federated data sources such as Amazon Redshift, Amazon DynamoDB, or Snowflake using SQL without extract, transform, and load (ETL) scripts. Now, data workers can connect to and unify these data sources within SageMaker Lakehouse. Federated source metadata is unified in SageMaker Lakehouse, where you apply fine-grained policies in one place, helping to streamline analytics workflows and secure your data.

Log into Amazon SageMaker Unified Studio, connect to a federated data source in SageMaker Lakehouse, and govern data with column- and tag-based permissions that are enforced when querying federated data sources with Athena. In addition to the SageMaker Unified Studio, you can connect to these data sources through the Athena console and API. To help you automate and streamline connector set up, the new user experiences allow you to create and manage connections to data sources with ease.

Now, organizations can extract insights from a unified set of data sources while strengthening security posture, wherever your data is stored. The unification and fine-grained access controls on federated sources are available in all AWS Regions where SageMaker Lakehouse is available. To learn more, visit SageMaker Lakehouse documentation .

Today, AWS announces the next generation of Amazon SageMaker, a unified platform for data, analytics, and AI. This launch brings together widely adopted AWS machine learning and analytics capabilities and provides an integrated experience for analytics and AI with unified access to data and built-in governance. Teams can collaborate and build faster from a single development environment using familiar AWS tools for model development, generative AI application development, data processing, and SQL analytics, accelerated by Amazon Q Developer, the most capable generative AI assistant for software development.

The next generation of SageMaker also introduces new capabilities, including Amazon SageMaker Unified Studio (preview), Amazon SageMaker Lakehouse, and Amazon SageMaker Data and AI Governance. Within the new SageMaker Unified Studio, users can discover their data and put it to work using the best tool for the job across data and AI use cases. SageMaker Unified Studio brings together functionality and tools from the range of standalone studios, query editors, and visual tools available today in Amazon EMR, AWS Glue, Amazon Redshift, Amazon Bedrock, and the existing Amazon SageMaker Studio. SageMaker Lakehouse provides an open data architecture that reduces data silos and unifies data across Amazon Simple Storage Service (Amazon S3) data lakes, Amazon Redshift data warehouses, and third party and federated data sources. SageMaker Lakehouse offers the flexibility to access and query data with Apache Iceberg–compatible tools and engines. SageMaker Data and AI Governance, including Amazon SageMaker Catalog built on Amazon DataZone, empowers users to securely discover, govern, and collaborate on data and AI workflows.  

For more information on AWS Regions where the next generation of Amazon SageMaker is available, see Supported Regions . 

To learn more and get started, visit the following resources:

• Amazon SageMaker product page
• Amazon SageMaker documentation
• Amazon SageMaker in the AWS Management Console

Amazon S3 Access Grants now integrate with AWS Glue for analytics, machine learning (ML), and application development workloads in AWS. S3 Access Grants map identities from your Identity Provider (IdP), such as Entra ID and Okta or AWS Identity and Access Management (IAM) principals, to datasets stored in Amazon S3. This integration gives you the ability to manage S3 permissions for end users running jobs with Glue 5.0 or later, without the need to write and maintain bucket policies or individual IAM roles.

AWS Glue provides a data integration service that simplifies data exploration, preparation, and integration from multiple sources, including S3. Using S3 Access Grants, you can grant permissions to buckets or prefixes in S3 to users and groups in an existing corporate directory, or to IAM users and roles. When end users in the appropriate user groups access S3 using Glue ETL for Apache Spark , they will then automatically have the necessary permissions to read and write data. S3 Access Grants also automatically update S3 permissions as users are added and removed from user groups in the IdP.

Amazon S3 Access Grants support is available when using AWS Glue 5.0 and later, and is available in all commercial AWS Regions where AWS Glue 5.0 and AWS IAM Identity Center are available. For pricing details, visit Amazon S3 pricing and Amazon Glue pricing . To learn more about S3 Access Grants, refer to the S3 User Guide .