Databases News, Latest Updates and Recent Announcements on Techminis

A naukri.com initiative

New

Home

Databases

Amazon

324

Image Credit: Amazon

Pre-warming Amazon DynamoDB tables with warm throughput

Amazon DynamoDB now offers warm throughput, providing insight into the read and write operations table or index can quickly support, with these values scaling as usage increases. Pre-warming a table is an asynchronous, non-blocking operation, allowing simultaneous executions of other table updates. The scaling capabilities of DynamoDB extend beyond the pre-warming activity, dynamically adjusting as the workload grows. Understanding and managing the current capabilities of a DynamoDB table is crucial, especially ahead of large-scale events. You can monitor warm throughput values using the DescribeTable API for both on-demand and provisioned mode tables.
DynamoDB has capacity modes where provisioned capacity mode allows you to set specific throughput ideal for predictable workloads. While on-demand mode scales automatically to meet demands which is suitable for unpredictable workloads. For more details, refer to DynamoDB throughput capacity in the developer guide.
Warm throughput values are not the maximum limit on your DynamoDB table’s capacity, rather it’s the minimum throughput that your table can handle instantaneously. Pre-warming is especially beneficial in scenarios with anticipated immediate traffic surges, such as product launches, flash sales, or major online events.
Before deciding whether to pre-warm your DynamoDB tables, you should estimate the peak throughput that your application might require. This helps ensure that the table is ready to handle traffic without throttling or performance problems. You can estimate this peak throughput by analyzing past traffic patterns or use forecasting skills.
Warm throughput provides a baseline of the number of reads and writes that a table can instantaneously support. This feature can be particularly beneficial in preparing a new on-demand table for high initial traffic, preparing for a migration to DynamoDB, and preparing for a large-scale event.
Warm throughput values are fully integrated into essential DynamoDB features, including global secondary indexes and global tables, supporting consistent performance across the entire system. One of the key advantages of warm throughput is its integration with infrastructure as code (IaC) tools such as AWS CloudFormation, making managing DynamoDB tables using IaC significantly more straightforward.
Pricing for this service is based on the cost of provisioned WCUs and RCUs in the specific Region where your table is located. By default, on-demand tables have a baseline warm throughput of 4,000 WCUs and 12,000 RCUs. When pre-warming a newly created table, you are only charged for the difference between your specified values and these baseline values, making it cost-effective.
Lee Hannigan, a Sr. DynamoDB Specialist Solutions Architect, brings a wealth of expertise in distributed systems, backed by a strong foundation in big data and analytics technologies. In his role as a DynamoDB Specialist Solutions Architect, Lee excels in assisting customers with the design, evaluation, and optimization of their workloads leveraging DynamoDB’s capabilities.

Read Full Article

19 Likes

Amazon

406

Image Credit: Amazon

Automate the deployment of Amazon RDS for Db2 Instances with Terraform

Amazon RDS for Db2 makes it easy to set up, operate, and scale Db2 deployments in the cloud.
Terraform, a popular Infrastructure as Code tool, can be used to deploy and manage RDS for Db2 instance.
Key properties to consider when deploying RDS for Db2 with Terraform are IBM Db2 version, database parameters, access control, data security, and monitoring features.
Amazon RDS for Db2 offers Multi-AZ deployment with disaster recovery solutions with cross-Region automated backups and security features.
The DB instance identifier serves as a way to identify and reference specific Db2 instance when interacting with the RDS API, AWS CLI commands, or the AWS Management Console.
Authentication of database users is provided either through password authentication or Kerberos authentication.
CloudWatch metrics, events, and enhanced monitoring can be used to monitor RDS for Db2 DB instance.
Amazon RDS for Db2 provides two encryption settings: encryption at rest and encryption in transit.
To enable encryption in transit with server identity verification for an RDS for Db2 DB instance it is required to assign a certificate authority (CA) to the DB instance among other steps.
Deploying Amazon RDS for Db2 involves careful consideration of various properties to achieve a secure, efficient, and high-performing database environment and Terraform can streamline the deployment process.

Read Full Article

24 Likes

Mysql

108

Image Credit: Mysql

Some InnoDB Cluster troubleshooting commands

To get the status of InnoDB Cluster, you can use the following commands:
To set the Primary Instance:
To obtain MySQL InnoDB Cluster basics:
To check for errors in the last 30 minutes, 4 hours, and 24 hours:
To reboot a failing node and perform rejoinInstance or addInstance:
To handle missing commands on replicas:
To delete an instance from the cluster and add it again:

Read Full Article

6 Likes

Dev

357

Image Credit: Dev

On-Demand Refresh | Materialized Views (Mviews) in ORACLE SQL

On-demand refresh allows users to manually trigger the refresh process for materialized views in Oracle SQL.
It provides more control over when the materialized view is refreshed and avoids the overhead of automatic refreshes.
On-demand refresh is useful for scheduled refreshes, performance control, integration with external systems, and resource optimization.
The DBMS_MVIEW.REFRESH procedure is used to perform an on-demand refresh, with options for complete or fast refresh methods.

Read Full Article

21 Likes

Dbi-Services

401

PostgreSQL: Maybe we should give ZFS a chance (2) – testing

In this post we’ll look at how they compare when it comes to performance and the data size on disk.
We’ve ended with two PostgreSQL clusters, one running on the ext4 file system while the other is running on top of the ZFS file system.
The size of the empty cluster is 39MB and 4.7MB on ext4 and ZFS file system respectively due to ZFS compression.
Loading data with pgbench shows the size for ext4 is 16GB while for ZFS is 914MB. The runtime was 382.58s for ext4 and 267.87s for ZFS.
The instance running on ZFS is slower than the instance running on ext4 for reading with simple pgbench tests.
pgbench test results for the same workload show ext4 is faster than the ZFS file system.
No clear winner between the two file systems, it depends on the workload and requirements.
Testing in your own environment is highly recommended.
ZFS might be worth considering for space on disk and snapshots and replication on the file system level.
ZFS might not be the best option for workloads that are mostly about reads.

Read Full Article

24 Likes

Amazon

135

Image Credit: Amazon

Understanding time-series data and why it matters

Time-series data is collected at regular or unpredictable intervals and tracks multiple data points over a specified time frame, providing a dynamic view of changes, patterns, and trends.
Time-series data enables predictive modeling and forecasting of future behaviors and values based on historical patterns and trends, and helps organizations predict spare part and resource demand based on sensor telemetry tracking wear rates of equipment components.
Anomaly detection is another important use case for time-series data, enabling organizations to identify anomalies or abnormalities by comparing new data points against established baselines and ranges based on past sequences.
Time-series data requires a dedicated store that can efficiently handle the high ingestion rates, automatic aggregation, and rapid querying of large datasets. Amazon Timestream offers two engine options to support your time-series needs.
Time-series data has become vitally important for many machine learning (ML) applications and models, as it captures patterns, sequences, and relationships that provide vital context.
Time-series data powers more accurate real-world algorithms and provides insights crucial for financial forecasting, demand planning, infrastructure monitoring, ML model efficiency, and predictive maintenance.
Victor Servin, Senior Product Manager for the Amazon Timestream team at AWS, brings 18 years of experience leading product and engineering teams in the Telco industry and an additional 5 years of expertise in supporting startups with Product Led Growth strategies and scalable architectures.
Servin's data-driven approach is suited to drive the adoption of analytical products like Timestream, and he helps customers to efficiently achieve their goals.
If you are already using a time-series engine like InfluxDB, check out the Migration Guide to speed up your move to manage journey, or explore Timestream engines.

Read Full Article

8 Likes

Amazon

162

Image Credit: Amazon

Use Amazon ElastiCache as a cache for Amazon Keyspaces (for Apache Cassandra)

This post explains how Amazon ElastiCache can be used as a write-through cache for read-intensive and cost-sensitive applications that store data in Amazon Keyspaces and may need submillisecond read response times.
The post uses a Cassandra Python client driver for accessing Amazon Keyspaces programmatically and a Redis client to connect to the ElastiCache cluster. The write-through caching strategy and lazy loading are used in the code provided in the post.
The post provides the following sample code snippets for Amazon Keyspaces operations, including for single-row INSERT and DELETE operations, retrieving a single book award by primary key, and retrieving a result set based on multiple parameters.
The post also suggests more options to cache data based on the nature of your access patterns, such as caching results based on partition key values only, sorting all key parameters in an order and hashing them, or sorting all query parameters and filters in an order and hashing them.
Implementing ElastiCache can help decrease read latency to submillisecond values, increase read throughput, and scale for higher loads without increasing costs for backend databases.
For more information and best practices on data modeling for Amazon Keyspaces, refer to the post linked within the summary.
Juhi Patil, a NoSQL Specialist Solutions Architect based in London, wrote the post to help customers design, evaluate, and optimize their Amazon Keyspaces and Amazon DynamoDB based solutions.

Read Full Article

9 Likes

Mysql

189

Image Credit: Mysql

Jemalloc install & config for MySQL

jemalloc is considered better than malloc for MySQL usage.
To install and configure jemalloc for MySQL:
Download the jemalloc package for your system from official sources or repositories.
Edit the mysqld.service file to preload jemalloc library and restart MySQL.
Verify that MySQL is using jemalloc.

Read Full Article

11 Likes

Pymnts

379

Image Credit: Pymnts

FIS and Oracle Team to Digitize Utility Bill Payments

FIS and Oracle are partnering to digitize utility bill payments, eliminating the need for paper checks.
FIS' BillerIQ solution, running on Oracle Cloud Infrastructure, will enable electronic bill delivery and digital payment acceptance for utility customers.
The utility industry still heavily relies on paper checks, with 75% of organizations using them for bill payments.
Using digital payment channels can streamline processes, lower days sales outstanding (DSO), and improve financial health.

Read Full Article

22 Likes

Global Fintech Series

447

Image Credit: Global Fintech Series

FIS and Oracle Enhance Utility Billing Experience

FIS and Oracle collaborate to enhance billing and payments capabilities for the utility industry.
FIS BillerIQ solution running on Oracle Cloud Infrastructure (OCI) will provide utility customers with flexible and tailored bill management features.
The collaboration aims to enable efficient and secure money flow from meter consumption to payment.
With options for digital payments, BillerIQ allows customers to manage utility bills according to their financial needs.

Read Full Article

26 Likes

Dbi-Services

140

Image Credit: Dbi-Services

Ace Your Certification Exams with Confidence: A Proven Recipe for Success

To pass certification exams with confidence, enhance your preparation, reduce exam-related stress, and avoid retakes.
Practical experience in the exam's focus area is crucial, as certifications validate expertise and hands-on experience.
Consult external and Microsoft documentation to deepen understanding of the exam material.
Practice with mock exams to familiarize yourself with the format and types of questions.

Read Full Article

8 Likes

Dev

371

Image Credit: Dev

Preventing SQL Injection (SQLi) Attacks in Drupal

SQL Injection (SQLi) is a common and potentially destructive security vulnerability that allows attackers to interfere with an application’s database.
Drupal, a popular CMS, is also susceptible to SQLi if not configured properly, making it essential for website administrators to take preventive measures.
Drupal has historically been targeted by SQLi attackers, highlighting the critical need for secure coding practices and frequent security audits.
To prevent SQLi in Drupal, it is important to use parameterized queries, update Drupal regularly, limit database privileges, and implement web application firewalls.

Read Full Article

22 Likes

Dev

Image Credit: Dev

UUID 🪪 or Auto Increment Integer / Serial ⚙️ as the Database Primary Key?

When designing a new SQL database schema, developers often need to choose between UUID or Auto Increment Integer/Serial as the primary key.
Auto Increment Integer/Serial is readable, occupies less space, but can't be used in distributed systems and may expose business data.
UUID is globally unique, stateless, and provides a sense of security, but is not readable and not naturally sortable by creation time.
UUIDv7 is a new UUID format that addresses some limitations by introducing time-ordering, improving performance and indexing.

Read Full Article

4 Likes

Amazon

Image Credit: Amazon

Building a GDPR compliance solution with Amazon DynamoDB

AWS Service Sector Industry Solutions has developed a feature that enables customers to efficiently locate and delete personal data upon request, helping them meet GDPR compliance requirements
The GDPR mandates organizations to obtain explicit consent before collecting personal data and provides individuals with the right to erasure.
The company needed a scalable, cost-effective solution to handle GDPR erasure requests.
The application manages extensive profile data across various services, and the process involved overcoming significant challenges related to data storage, retrieval, and deletion while also minimizing disruption to customers’ operations.
One of the primary design challenges was efficiently locating and purging profile data stored in Amazon S3, especially considering the terabytes of data involved.
For GDPR erasure of profile data in Amazon S3, the team built a custom solution predominantly using the Go programming language and aLambda function using AWS SDK for Pandas in Python.
The use of Parquet, a columnar storage format, allows Athena to query only the necessary columns rather than entire rows, as required with CSV files.
To achieve distributed mutex using DynamoDB, they used a custom mutex client.
The mutex client uses DynamoDB ConditionExpression to make sure the RVN has not changed from what was previously stored.
This solution can be adapted for other use cases requiring secure, distributed locking mechanisms or efficient data management across large datasets.

Read Full Article

5 Likes

Amazon

171

Image Credit: Amazon

Heterogenous data sources: Access your data in PostgreSQL from Amazon RDS for Oracle using Oracle Database Gateway

Amazon RDS for Oracle databases need to connect to external data sources, such as RDS for PostgreSQL. PostgreSQL can establish connections to Oracle databases using a foreign data wrapper (FDW). However, when connecting from RDS for Oracle to PostgreSQL, you need an Oracle Database Gateway for ODBC (DG4ODBC) installed on a separate server. In this post, we walk you through setting up an EC2 instance as a database gateway server.
For RDS for Oracle databases to connect to external data sources, you need to install Oracle Database Gateway on an EC2 instance along with the required ODBC drivers.
The following diagram provides an overview of the solution. In RDS for Oracle, when a user needs to access data from RDS for PostgreSQL, the solution uses a database link pointing to Oracle Database Gateway for ODBC (DG4ODBC), which uses ODBC drivers and PostgreSQL client libraries to connect to RDS for PostgreSQL and retrieves the data.
Create an EC2 instance with Linux x86-64 OS, which will be used to install and run Oracle Database Gateway for ODBC.
Make sure the EC2 instance meets the minimum hardware and software requirements to run Oracle Database Gateway for ODBC.
Update the Oracle Database Gateway for ODBC EC2 instance’s security group to allow inbound traffic from the RDS for Oracle instance.
Update the RDS for PostgreSQL security group to allow inbound traffic from the Oracle Database Gateway for ODBC EC2 instance.
Prepare wget.sh script to download the Oracle Database Gateway software.
Install the required packages and unzip the install file.
Install the Unix ODBC Driver Manager and the latest PostgreSQL client, libraries and ODBC drivers.

Read Full Article

10 Likes

For uninterrupted reading, download the app

Databases

Discover more