A naukri.com initiative
Big Data News
Siliconangle
1M
338
Image Credit: Siliconangle
Alteryx simplifies analytics for hybrid data infrastructures
- Alteryx is expanding its platform to support hybrid data infrastructures.
- The Fall 2024 update includes new AI capabilities to automate insight generation and streamline reporting.
- The update introduces connectors for Google Cloud Storage and SingleStore, making it easier to set up data pipelines.
- Other enhancements include Magic Reports for advanced data reporting and visualization, as well as improvements to data preparation and blending tools.
Read Full Article
20 Likes
Amazon
1M
429
Image Credit: Amazon
Use Amazon Kinesis Data Streams to deliver real-time data to Amazon OpenSearch Service domains with Amazon OpenSearch Ingestion
- In this post, we show how to use Amazon Kinesis Data Streams to buffer and aggregate real-time streaming data for delivery into Amazon OpenSearch Service domains and collections using Amazon OpenSearch Ingestion.
- Kinesis Data Streams enhances log aggregation by decoupling producer and consumer applications, and providing a resilient, scalable buffer to capture and serve log data.
- OpenSearch Ingestion is a serverless pipeline that provides powerful tools for extracting, transforming, and loading data into an OpenSearch Service domain.
- The use case for centralizing log aggregation is also discussed for an organization that has a compliance need to archive and retain its log data, and how standardizing logging approaches reduces development and operational overhead for organizations.
- The article guides readers through creating an AWS Identity and Access Management (IAM) role that allows read access to the Kinesis data stream and read/write access to the OpenSearch domain for configuring OpenSearch Ingestion pipeline to process log data and providing detailed explanation with example how to parse the log message fields.
- Readers are also provided with several key areas to monitor for maintaining the health of log ingestion pipeline such as Kinesis Data Streams metrics, CloudWatch subscription filter metrics, OpenSearch Ingestion metrics, and OpenSearch Service metrics.
- Lastly, the article concludes with some suggestions for other use cases for OpenSearch Ingestion and Kinesis Data Streams, such as using anomaly detection, trace analytics, and hybrid search with OpenSearch Service.
- The authors of the article are M Mehrtens, Arjun Nambiar, and Muthu Pitchaimani.
Read Full Article
25 Likes
Amazon
1M
248
Image Credit: Amazon
Achieve data resilience using Amazon OpenSearch Service disaster recovery with snapshot and restore
- Amazon OpenSearch Service is a fully managed service offered by AWS that enables you to deploy, operate, and scale OpenSearch domains effortlessly.
- Snapshot and restore in OpenSearch Service involves creating point-in-time backups, known as snapshots, of your OpenSearch domain.
- Snapshot and restore strategy helps organizations meet Recovery Point Objectives (RPOs) and Recovery Time Objectives (RTOs).
- The snapshot and restore strategy in OpenSearch Service results in longer downtimes and greater loss of data between when the disaster event occurs and recovery.
- In the event of a disaster, you can fail over to the OpenSearch Service domain in the secondary Region using IaC.
- Infrastructure as code (IaC) methods such as using AWS CloudFormation or the AWS Cloud Development Kit (AWS CDK) enable you to deploy consistent infrastructure across Regions.
- In a disaster, if your OpenSearch Service domain in the primary Region goes down, you can fail over to a domain in the secondary Region.
- To maintain business continuity during a disaster, you can use message queues like Amazon Simple Queue Service (Amazon SQS) and streaming solutions like Apache Kafka or Amazon Kinesis.
- When the primary Region becomes available again, you can seamlessly revert to the OpenSearch Service domain in the primary Region.
- In conclusion, by following the best practices provided in the AWS Well-Architected Reliability Pillar, you can achieve data resilience using Amazon OpenSearch Service disaster recovery with snapshot and restore.
Read Full Article
14 Likes
Precisely
1M
167
Image Credit: Precisely
Boosting Customer Loyalty with Personalization and Communication Strategies
- Personalized customer experiences are key to building customer loyalty.
- Data-driven personalization analyzes customer behaviour to tailor communications to meet specific needs.
- High-quality data is necessary for effective personalization.
- Data-driven personalization improves customer experience, engagement and loyalty.
- To implement data-driven personalization, businesses should focus on high-quality data, develop a personalization strategy, and measure and optimize efforts.
- Siloed communication channels create fragmented customer experiences, leading to poor customer satisfaction.
- Unified communication channels streamline change management, improve collaboration and enhance customer engagement.
- Choosing the right transformation partner is essential, including deep technical expertise, proven change management skills, and experience in customer experience management.
- Improving data-driven personalization and unified communications increases customer loyalty, streamlines operations and drives sustained success in the competitive market.
- Partnering with a digital transformation expert is a crucial step in successfully navigating the journey to stronger CX.
Read Full Article
10 Likes
Precisely
1M
438
Image Credit: Precisely
4 Practical Tips for Implementing Data-Driven Personalization
- Data-driven personalization involves using your customer data to tailor communications and interactions that meet their individual preferences and needs.
- The data used for personalization must be of high quality, accurate, up-to-date, and free of redundancies to avoid inconsistent messaging, which erodes customer trust.
- Implementing a unified customer communication solution is crucial to avoid fragmented customer experiences and inconsistent messaging due to siloed communication channels.
- Unified communications streamline change management, improve collaboration, and eliminate redundant processes, leading to faster decision-making and a more agile approach to customer engagement.
- To implement data-driven personalization and unify communications, businesses need a digital transformation expert who can integrate advanced technologies and manage organizational change.
- Selecting the right partner with deep technical expertise, proven change management skills, and customer experience management (CXM) experience is essential for success.
- Data-driven personalization benefits businesses by enhancing the customer experience, increasing customer engagement, and improving customer loyalty.
- Specific tips for implementing data-driven personalization include unifying communication channels, focusing on high-quality data, developing a personalization strategy, and measuring and optimizing efforts.
- High-quality data and seamless communication across all touchpoints are essential for driving engagement and building long-term customer loyalty.
- Partnership with a digital transformation expert is crucial to navigating the journey of data-driven personalization and unified communications for businesses.
Read Full Article
26 Likes
Amazon
1M
417
Image Credit: Amazon
Incremental refresh for Amazon Redshift materialized views on data lake tables
- Amazon Redshift allows precomputed query results in the form of materialized views for faster query response times from your data warehouse.
- Redshift supports incremental refresh capability for local tables, which is useful for aggregations and multi-table joins specifically.
- Customers use data lake tables for cost-effective storage and interoperability with other tools.
- Amazon Redshift now provides the ability to incrementally refresh installed materialized views on data lake tables.
- Incremental refreshes on standard data lake tables enable building and refreshing materialized views in Amazon Redshift maintaining data freshness with a cost-effective approach.
- Incremental refreshes are also possible for data lake tables using Apache Iceberg.
- Amazon Redshift's introduction of incremental refresh provides substantial performance gains over full recompute.
- Materialized views on data lake tables can be valuable for optimizing SQL queries for faster data analysis.
- For best practices on materialized views on data lake tables in Amazon Redshift Spectrum, check out the AWS documentation.
- Amazon Redshift makes it cost-effective to analyze structured and semi-structured data using standard SQL and business intelligence tools.
Read Full Article
25 Likes
Amazon
1M
45
Image Credit: Amazon
Amazon OpenSearch Service announces Standard and Extended Support dates for Elasticsearch and OpenSearch versions
- Amazon OpenSearch Service supports 19 versions of Elasticsearch opensource, and 11 versions of OpenSearch.
- End of Support dates for legacy Elasticsearch versions up to 6.7, Elasticsearch versions 7.1 through 7.8, OpenSearch versions from 1.0 through 1.2, and OpenSearch versions 2.3 through 2.9 available on Amazon OpenSearch Service, have been announced.
- We recommend that customers running Elasticsearch versions upgrade to the latest OpenSearch versions.
- All Elasticsearch versions will receive at least 12 months of Extended Support.
- For OpenSearch versions running on Amazon OpenSearch Service, we will provide at least 12 months of Standard Support after the end of support date for the corresponding upstream open source OpenSearch version.
- Upgrading your domain to the latest available OpenSearch version will help you derive maximum value out of OpenSearch Service.
- Domains running versions under Extended Support will be charged an additional fee per normalized instance hour.
- We add new capabilities across various vectors to the latest OpenSearch versions, which include new features, performance and resiliency improvements, and security improvements.
- For any questions on Standard and Extended Support options, see the FAQs. For further questions, contact AWS Support.
- The authors of this announcement are Arvind Mahesh, Kuldeep Yadav, and Jon Handler.
Read Full Article
2 Likes
Siliconangle
1M
408
Image Credit: Siliconangle
Uber revamps its operating model with real-time data and microservices orchestration
- Uber has built an expansive architecture that powers real-time capabilities such as logistics services and ticket bookings.
- Central to Uber’s application is its real-time nature, a fundamental aspect that sets it apart from many other applications.
- The real-time interactions demand that the server rather than the app initiate much of the data push.
- Uber's infrastructure combines various technologies to handle complex processes.
- The company uses Google Spanner as the transactional database and custom frameworks for real-time event propagation and orchestration.
- Behind-the-scenes, machine learning algorithms facilitate the dynamism of Uber’s different apps.
- The data generated by Uber’s users and server-side data need to be meticulously synchronized.
- Events and metadata are sent into Kafka for processing and storage in Uber’s Hive tables.
- The app’s logic is informed by both current context and historical patterns.
- Real-time orchestration and machine learning integration are essential to the smooth functioning of Uber’s different apps.
Read Full Article
24 Likes
Amazon
1M
313
Image Credit: Amazon
Amazon OpenSearch Service launches the next-generation OpenSearch UI
- Amazon OpenSearch Service has launched a modernised analytical service, delivering insights from OpenSearch and other integrated data sources.
- A new OpenSearch workspace will curate analytics experiences and support access control assisting with private/internal use cases.
- The new Discover feature enables multiple data sources, natural language query generation and SQL and Piped Processing Language, simplifying interactive analysis.
- The Workspace management tool helps manage configurations and permissions, so users can analyse data without switching tools, reducing complexity and increasing efficiency.
- Users can work with multiple OpenSearch clusters and scale workloads across multiple clusters to analyse data in one place using the new UI.
- Improved filtering and navigation in the Workspace improves the user experience and reduces time spent searching through menus making it more productive.
- The ENH repeated programming language support means diverse users can grow insights from their data naturally.
- The service is available in North Virginia, Oregon, North California, Mumbai, Singapore, Sydney, Tokyo, Paris, Frankfurt, Ireland, Sao Paolo, and Canada Central regions.
- Amazon OpenSearch Service helps leverage cloud technologies and building data products that assist users with actionable insights and achieve operational excellence.
- The updates ensure that OpenSearch offers a modern, easy-to-use analytics experience for managing and analysing operational data with authorised access.
Read Full Article
18 Likes
TechBullion
1M
308
Image Credit: TechBullion
Transforming Energy Sector Supply Chains: A Deep Dive with Paula Gonzalez on Machine Learning and Digital Innovation
- The intersection of machine learning and supply chain management is fundamentally reshaping how energy companies approach procurement, logistics, and operational efficiency.
- Currently pursuing her MBA at Rice University’s Paula Gonzalez speaks with us in an in-depth interview.
- Paula has been at the forefront of digital transformation initiatives, implementing enterprise-wide procurement platforms and developing innovative analytics solutions.
- She explores how machine learning is revolutionizing supply chain processes, shares strategies for successful digital adoption, and provides a forward-looking perspective on the future of supply chain optimization.
- Machine learning models provide supply chain practitioners with more accurate forecasts and identify cost-savings opportunities by analyzing historical and real-time data.
- Predictive analytics has revolutionized industrial operations by providing more accurate demand forecasting which can be translated into cost-reduction opportunities.
- Data accessibility, real-time updates, and customizable views in developing dashboards translate into practical advantages for supply chain operations.
- Driving digital adoption in an enterprise setting requires addressing the cultural shift just as much as the technical integration.
- Maintaining the balance between advanced technology and human expertise in supply chain operations is important and it requires three strategies.
- Automated contract management systems and logistics algorithms are set to become increasingly sophisticated.
Read Full Article
18 Likes
Amazon
1M
335
Image Credit: Amazon
Build up-to-date generative AI applications with real-time vector embedding blueprints for Amazon MSK
- Generative AI has become a focal point for creating intelligent applications that deliver personalized experiences, but static pre-trained models often struggle to provide accurate and up-to-date responses without real-time data.
- To address this, real-time vector embedding blueprints have been introduced that simplify building real-time AI applications by automatically generating vector embeddings using Amazon Bedrock from streaming data in Amazon Managed Streaming for Apache Kafka (Amazon MSK) and indexing them in Amazon OpenSearch Service.
- This enables Retrieval Augmented Generation (RAG) capabilities for generative AI models by ingesting streaming data, generating vector embeddings, and storing them in a vector database for later retrieval. RAG optimizes the output of an LLM so it references an authoritative knowledge base outside its training data sources to ensure accurate results.
- Traditional LLMs are often limited by their reliance on static information, leading to outdated or irrelevant responses. Integrating real-time data streams helps ensure that generative AI applications provide contextually rich, up-to-date information to deliver accurate, reliable, and meaningful responses to end users.
- The real-time vector embedding blueprint automates the generation of vector embeddings from streaming data, storing them in a vector database, and makes the data available for generative AI applications to query and process. This simplifies the development process, allowing teams to focus on innovation and improving their AI applications.
- By integrating streaming data ingestion, vector embeddings, and RAG techniques, organizations can enhance the capabilities of their generative AI applications. Real-time vector embedding blueprints further simplify the development process, allowing businesses to remain agile, responsive, and innovative.
- Real-time vector embedding blueprints are available in several AWS regions, enabling businesses to build real-time AI applications that deliver personalized experiences and optimize user engagement.
- The authors of this article work with AWS customers to help them design scalable and efficient streaming architectures using Amazon MSK and Amazon Managed Service for Apache Flink.
Read Full Article
20 Likes
Amazon
1M
363
Image Credit: Amazon
Reduce your compute costs for stream processing applications with Kinesis Client Library 3.0
- Kinesis Client Library (KCL) enables you to reduce your stream processing cost by up to 33% compared to previous KCL versions by redistributing the load evenly to all workers using a new load balancing algorithm
- KCL uses Kinesis Data Streams APIs to read data from the streams and handles the heavy lifting of balancing stream processing across multiple workers, managing failovers and checkpointing processed records
- Customers processing real-time data streams typically use multiple compute hosts such as Amazon Elastic Compute Cloud (Amazon EC2) to handle the high throughput in parallel
- With KCL 3.0, uneven load distribution across workers that results in higher costs can be addressed
- KCL 3.0 introduces a new load balancing algorithm that continuously monitors the resource utilization of workers and redistributes the load evenly to all workers
- Additionally, KCL 3.0 offers several other benefits including Amazon DynamoDB read capacity unit (RCU) reduction, Graceful handoff of shards from one worker to another, and Removal of the AWS SDK for Java 1.x dependency
- You don’t have to make any changes to your application code to migrate from KCL 2.x version to KCL 3.0
- We recommend checking the IAM permissions required for KCL consumer applications, maxLeasesForWorker configuration, and scaling policy after upgrading to KCL 3.0
- KCL 3.0 introduces significant enhancements that can help optimize the cost-efficiency and performance of KCL applications
- The new load balancing algorithm enables more even CPU utilization across worker instances, potentially allowing for right-sized and more cost-effective stream processing fleets
Read Full Article
21 Likes
Amazon
1M
27
Image Credit: Amazon
Stream real-time data into Apache Iceberg tables in Amazon S3 using Amazon Data Firehose
- Engineering teams are increasingly replacing batch data processing pipelines with real-time streaming, and building data lakes to store their data, adopting open data formats such as Parquet and Apache Iceberg to store their data.
- This trend is being seen across many industries such as online media, gaming companies, factories monitoring equipment for maintenance and failure, theme parks providing wait times for popular attractions.
- Apache Iceberg is becoming popular among customers storing their data in Amazon S3 data lakes, because it allows customers to read and write data concurrently using different frameworks.
- Amazon Data Firehose simplifies the process of streaming data by allowing users to configure a delivery stream, select a data source, and set Iceberg tables as the destination.
- Firehose is integrated with over 20 AWS services, and supports routing data to different Iceberg tables to have data isolation or better query performance.
- This post describes how to set up Firehose to deliver data streams into Iceberg tables on Amazon S3, and addresses different scenarios for routing data into iceburg tables.
- For instance, routing records to different tables based on the content of the incoming data by specifying a JSON Query expression can be accomplished by setting the 'Database expression' and 'Table expression' fields.
- Alternatively, routing records to different tables based on the content of the incoming data can be achieved by using a Lambda function, as described in use case 4.
- All of the AWS services used in these examples are serverless, and no infrastructure management is required.
- Users can query data they’ve written to Iceberg tables using different processing engines such as Apache Spark, Apache Flink, or Trino, or use Amazon Athena.
Read Full Article
1 Like
Precisely
1M
304
Image Credit: Precisely
Women on Wednesday with Sierra Weltha
- Sierra Weltha, Associate Product Marketing Manager at Precisely, has revealed why she chose a technology career, who her mentors have been, and how they have helped her.
- Advocating for women in tech, Precisely’s Women in Technology (PWIT) network provides a space for women in the organization to connect and learn from one another.
- Diverse teams in the technology industry, which includes more women than ever, perform better, according to CIO magazine. The gender gap still needs to be bridged.
- Sierra has found that imposter syndrome has been one of the challenges of being a woman in tech and, like everyone else, has faced occasions where she has wondered whether her gender had played a role in certain situations.
- She has learned that self-conviction is important, with lessons from her mentors providing her with clarity and resilience in times of challenge.
- Sierra has taken risks throughout her career, including changing jobs, exploring new industries and recognising when to say no to certain opportunities.
- Her best advice was to understand herself and focus on what aligns with her goals and values, while personality assessments have helped her gain perspective.
- Sierra encourages younger women entering the industry to have confidence, commit to growth and take risks, and to share positivity and confidence with others.
- She pays tribute to those who support and encourage her, celebrating the accomplishments, following up on conversations and participating in groups such as Precisely Women in Technology.
- Sierra believes that success is far more rewarding when it is shared with others and hopes to create an environment where women feel empowered to reach their fullest potential.
Read Full Article
18 Likes
Siliconangle
1M
231
Image Credit: Siliconangle
Data-driven storytelling: Ingka Group’s path to operational excellence
- Ingka Group is leveraging data-driven insights to optimize customer order flows and enhance process efficiency.
- The company started engaging with Celonis over three years ago to improve transparency and make data-driven conclusions.
- Striving for a seamless customer experience, Ingka Group focuses on preventing imperfections and enhancing efficiency.
- Data is used to create narrative storytelling and inspire change within the organization.
Read Full Article
13 Likes
For uninterrupted reading, download the app