Implementation of Data Archival Solution with GenAI

Organizations are now modernizing their legacy systems by creating automated, AI-enabled platforms. One major challenge is handling the efficient data preparation and archival absence of which leads to data loss, skewness , and costlier reporting and analytics. Wipro, an AWS Premier Consulting Partner and Managed Service Provider (MSP), addresses these challenges by delivering cloud data archival solutions using GenAI. GenAI in AWS AWS Bedrock is a fully managed GenAI service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, DeepSeek, Luma, Meta, Mistral AI, poolside (coming soon), Stability AI, and Amazon through a single API. It provides a broad set of capabilities needed to build generative AI applications with security, privacy, and responsible AI. Using Amazon Bedrock, you can easily experiment with and evaluate top FMs for your use case, privately customize them with your data using techniques such as fine-tuning and Retrieval Augmented Generation (RAG), and build agents that execute tasks using your enterprise systems and data sources. AWS Bedrock can be used to generate code in various stages of the software development lifecycle (SDLC). allows developers to create their own systems to augment, write, and audit code by using models within Amazon Bedrock instead of relying on out-of-the-box coding assistants. Code interpretation in Amazon Bedrock enables your agent to generate, run, and troubleshoot your application code in a secure test environment. This includes tasks such as understanding user requests for specific tasks, generating code to perform those tasks, executing the code, and providing the result from the code execution. Solution Overview- The solution we have proposed is using AWS bedrock service which will generate the automated scripts based on the prompt provided by the user. Data Flow- 1) We will use AWS native connectors to connect through customer source systems. Data will be stored in S3.Based on the event trigger of full load file or CDC accordingly AWS Glue or DMS will be triggered and data will be copied over to S3 raw layer. 2) The data will be catalogued and processed using AWS Glue and will be stored in processed layer. Post which data lifecycle policy will be implemented on the bucket for data archival. 3) Bedrock model will be invoked using boto3 for code generation wherever required. 4) Metadata of the data archival will be available for query and report generation. 5) Based on reporting user requirement through Quicksight, Lambda will be used to invoke the model and generate required query for the data which will be serviced via athena. The various capabilities of our solution are:- 1) Ability to effectively perform the script generation activities using AWS native GenAI services. 2) Increased Reusability of the script used for result generation from the models. 3) Auto optimized script generation from GenAI. 4) Cost Effective archieval solution based on serverless architecture. 5) Automated archival framework will provide fully integrated skeleton for reuse. 6) Efficiency and effectiveness in script preparation. 7) Event based trigger for pipeline creation and processing. 8) Tight coupling with all AWS native services. 9) Less manual intervention in the fully integrated solution. 10) End to End data delivery using cloud agnostic solution provide scalability and cost effectiveness. Benefits- 1) Process Efficiency: Increases overall efficiency in script generation process upto 90% 2) Effort optimization: Up to 40% reduction in involvement of in-house teams required for data archieval activities. 3) Reduction in the requirement for proficient and highly skilled people. 4) Proper organized data and its metadata availability to give wider view to the users. Industrial usage- The Data Archival with GenAI as a solution has benefits across industries as efficient data preparation is required by most of the industry process for operational functions. E.g., for Retail industry monthly sales analysis, for health care it could be medical records used for future prediction of upcoming health challenges, for Finance industry finding out the fund utilization rate in real time, etc. So the overall solution will deliver cloud transformation at scale with GenAI in a speedy manner needed for most of the organizations and implementing the solution leveraging Amazon Cloud services.

May 2, 2025 - 14:54

Implementation of Data Archival Solution with GenAI

Organizations are now modernizing their legacy systems by creating automated, AI-enabled platforms. One major challenge is handling the efficient data preparation and archival absence of which leads to data loss, skewness , and costlier reporting and analytics.
Wipro, an AWS Premier Consulting Partner and Managed Service Provider (MSP), addresses these challenges by delivering cloud data archival solutions using GenAI.

GenAI in AWS
AWS Bedrock is a fully managed GenAI service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, DeepSeek, Luma, Meta, Mistral AI, poolside (coming soon), Stability AI, and Amazon through a single API. It provides a broad set of capabilities needed to build generative AI applications with security, privacy, and responsible AI. Using Amazon Bedrock, you can easily experiment with and evaluate top FMs for your use case, privately customize them with your data using techniques such as fine-tuning and Retrieval Augmented Generation (RAG), and build agents that execute tasks using your enterprise systems and data sources.
AWS Bedrock can be used to generate code in various stages of the software development lifecycle (SDLC). allows developers to create their own systems to augment, write, and audit code by using models within Amazon Bedrock instead of relying on out-of-the-box coding assistants.
Code interpretation in Amazon Bedrock enables your agent to generate, run, and troubleshoot your application code in a secure test environment. This includes tasks such as understanding user requests for specific tasks, generating code to perform those tasks, executing the code, and providing the result from the code execution.

Solution Overview-
The solution we have proposed is using AWS bedrock service which will generate the automated scripts based on the prompt provided by the user.

Data Flow-
1) We will use AWS native connectors to connect through customer source systems. Data will be stored in S3.Based on the event trigger of full load file or CDC accordingly AWS Glue or DMS will be triggered and data will be copied over to S3 raw layer.
2) The data will be catalogued and processed using AWS Glue and will be stored in processed layer. Post which data lifecycle policy will be implemented on the bucket for data archival.
3) Bedrock model will be invoked using boto3 for code generation wherever required.
4) Metadata of the data archival will be available for query and report generation.
5) Based on reporting user requirement through Quicksight, Lambda will be used to invoke the model and generate required query for the data which will be serviced via athena.
The various capabilities of our solution are:-
1) Ability to effectively perform the script generation activities using AWS native GenAI services.
2) Increased Reusability of the script used for result generation from the models.
3) Auto optimized script generation from GenAI.
4) Cost Effective archieval solution based on serverless architecture.
5) Automated archival framework will provide fully integrated skeleton for reuse.
6) Efficiency and effectiveness in script preparation.
7) Event based trigger for pipeline creation and processing.
8) Tight coupling with all AWS native services.
9) Less manual intervention in the fully integrated solution.
10) End to End data delivery using cloud agnostic solution provide scalability and cost effectiveness.

Benefits-
1) Process Efficiency: Increases overall efficiency in script generation process upto 90%
2) Effort optimization: Up to 40% reduction in involvement of in-house teams required for data archieval activities.
3) Reduction in the requirement for proficient and highly skilled people.
4) Proper organized data and its metadata availability to give wider view to the users.

Industrial usage-
The Data Archival with GenAI as a solution has benefits across industries as efficient data preparation is required by most of the industry process for operational functions. E.g., for Retail industry monthly sales analysis, for health care it could be medical records used for future prediction of upcoming health challenges, for Finance industry finding out the fund utilization rate in real time, etc. So the overall solution will deliver cloud transformation at scale with GenAI in a speedy manner needed for most of the organizations and implementing the solution leveraging Amazon Cloud services.