AWS Glue adds new transforms (Purge, Transition and Merge) for Apache Spark applications to work with datasets in Amazon S3

AWS Glue now supports three new transforms – Purge, Transition, Merge – that can help you extend your extract, transform, and load (ETL) logic in Apache Spark applications. You can use the Purge transform to remove files, partitions or tables, and quickly refine your datasets on S3.