September 1, 2025

AWS DynamoDB Shell - ddbsh

DynamoDB Shell (ddbsh) commands in AWS


An open-source command line interface (CLI) and interactive shell to run SQL-like DDL and DML commands on Amazon DynamoDB (DDB). And these SQL commands will be automatically translated to DynamoDB queries. DynamoDB Shell is written in C++ and uses the DynamoDB API through the AWS DynamoDB SDK.

For installation and building the DynamoDB Shell tool, please refer to the project’s GitHub page.

$ ddbsh
help update;
help;
HELP - provide help in ddbsh

   HELP <keyword> [keyword [keyword ...]]

      Provides help about the specified keyword, or statement.

      HELP ALTER TABLE
      HELP BACKUP
      HELP BEGIN
      HELP COMMIT
      HELP CONNECT
      HELP CREATE
      HELP DELETE
      HELP DESCRIBE BACKUP
      HELP DESCRIBE
      HELP DROP BACKUP
      HELP DROP TABLE
      HELP DROP
      HELP EXPLAIN
      HELP INSERT
      HELP REPLACE
      HELP RESTORE
      HELP ROLLBACK
      HELP SELECT
      HELP SHOW BACKUPS
      HELP SHOW CREATE TABLE
      HELP SHOW LIMITS
      HELP SHOW VERSION
      HELP SHOW
      HELP UPDATE
      HELP UPSERT

connect us-east-2;
show version;
show limits;
show tables;
begin;
commit;
rollback;
describe dynamodb_tab;
show create table ddbsh_demo;

CREATE TABLE commands in DynamoDB shell

CREATE TABLE [IF NOT EXISTS][NOWAIT] <name>  ( attribute_name, attribute_type [,...] )
   primary_key billing_mode_and_throughput
   [gsi_list] [lsi_list] [streams] [table_class] [tags] ;
create table ddbsh_demo (id number) primary key (id hash);
create table ddbsh_demo2 (pk number, rk number) primary key (pk hash, rk range);
create table ddb_tab ( a number, b number ) primary key ( a hash ) gsi ( gsione on (b hash) projecting all);
create table ddb_table ( id string, name string ) primary key ( id hash ) billing mode provisioned ( 5 rcu, 5 wcu ) gsi ( namegsi on (name hash) projecting all billing mode provisioned ( 5 rcu, 5 wcu ));
create table if not exists nowait ddbsh_demo3 (id string, accttype string, balance number) primary key (id hash, accttype range) billing mode provisioned (20 rcu, 20 wcu) gsi (balancegsi on (accttype hash, balance range) projecting all billing mode provisioned (20 rcu, 20 wcu)) stream (both images);

ALTER TABLE commands on DynamoDB tables using DDB shell

alter table dynamodb_tab set billing mode on demand;
alter table dynamodb_tab set billing mode provisioned (200 RCU, 300 WCU);
alter table dynamodb_tab set table class standard infrequent access;
alter table dynamodb_tab set table class standard;
alter table dynamodb_tab set stream (new image);
alter table dynamodb_tab set stream disabled;
alter table dynamodb_tab set pitr enabled;
alter table dynamodb_tab set pitr disabled;
alter table dynamodb_tab set deletion protection disabled;
alter table dynamodb_tab set deletion protection enabled;
alter table ddb_tab (x number, y number) create gsi xygsi on (x hash, y range) projecting all billing mode provisioned (20 RCU, 40 WCU);
alter table ddb_tab (x number, y number) create gsi xykeysgsi on (x hash, y range) projecting keys only billing mode provisioned (22 RCU, 41 WCU);
alter table ddb_tab (x number, y number, a number) create gsi xyagsi on (x hash) projecting include (y, a) billing mode provisioned (25 RCU, 50 WCU);
alter table ddb_tab set billing mode provisioned (20 rcu, 30 wcu) update gsi (pqgsi set billing mode provisioned (20 rcu, 30 wcu), xykeysgsi set billing mode provisioned (3 rcu, 5 wcu));
alter table ddb_tab drop gsi xygsi;
alter table ddbsh_demo add replica us-east-2;
alter table ddbsh_demo add replica us-west-2 table class standard infrequent access;
alter table ddbsh_demo drop replica us-east-2;
alter table ddbsh_demo set ttl (created_epoch_time);
alter table ddbsh_demo set ttl disabled;

If you want to add new column(s), there is NO command like alter table table-name add column column-name, you have to use INSERT or UPDATE or UPSERT commands with values for new columns (along with existing columns).

DROP TABLE commands on Amazon DynamoDB tables

drop table ddbsh_demo;
drop table if exists ddb_tab;

SELECT queries/commands on DynamoDB tables using DynamoDB shell

SELECT [CONSISTENT] attribute_list | * FROM <table>[.<index>] [WHERE where_clause] [return_clause | ratelimit] ]
select * from ddbsh_demo;
select * from ddbsh_demo where string_col = "available";
select * from ddbsh_demo where a = 5;
select id, a from ddbsh_demo where a > 3 and a < 5;
select * from ddbsh_demo where a between 2 and 5;
select * from ddbsh_demo where rangekey in (3, 9, 4, 7);
select consistent * from ddbsh_demo where attribute_exists(c) and a != 8;
select * from ddbsh_demo where attribute_exists(c) or a = 33;
select * from ddbsh_demo where attribute_exists(c);
select * from ddbsh_demo where not attribute_exists(amount);
select * from ddbsh_demo where attribute_type(b, string);
select * from ddbsh_demo where attribute_type(id, number) return total;
select * from ddbsh_demo where begins_with(b, "co");
select * from ddbsh_demo where contains(pk, "at");
select a, b, c from ddbsh_demo where attribute_type(b, string) or (a = 5 and c = "coffee");
select * from ddbsh_demo where size(id) < 5;
select * from ddbsh_demo where v.c = true;
select * from ddbsh_demo where v.b[1] = 11;
select * from ddbsh_demo.gsi1;
select * from ddb_tab.gsi2 where name = "Brutus";
select * from ddbsh_demo where a between 2 and 5 WITH RATELIMIT (5 RCU);
select * from ddbsh_demo where a between 2 and 5 WITH RATELIMIT (2 RCU, 2 WCU);

INSERTing records INTO DDB tables with DynamoDB shell

insert into ddbshell (custid, name) values (103, "Charlie");
insert into ddbshell (custid, name) values (101, "Alice"), (102, "Bob");
insert into ddbshell (pk, rk, x, y, z) values (4, "three", 11, 12, 13);
insert into ddbshell (id, v) values (3, 4), (4, "a string value"), (5, {a: 4, b: [10, 11, 12], c: true, d: {x: 10, y: 10}});
insert into ddb_tab (a, b) values ( 1, 2 ), (2, 3), (3, 4) with ratelimit ( 2 wcu );

REPLACE INTO <table> ( column [, column ...] ) VALUES ( values ) [ratelimit]
replace into ddbshell (pk, rk, ins) values (12, "nonexistant", "inserted");

UPDATE commands on Amazon DynamoDB tables using DDBShell

update ddbsh_demo set b = 15 where a = 5;
update ddb_tab age= 14 where name = "Brutus";
update ddbsh_demo set active = True where custid = 111;
update ddbsh_demo set newattr = 14 where pk = 1 and rk = "one";
update ddbsh_demo set newattr = if_not_exists(y, 3) + 6 where pk = 1;
update ddb_tab set balance = balance + 100 where id = "Satya" and accttype = "Savings";
update ddbsh_demo set z = 14, v.b[1] = 13 where id = 5;
update ddb_tab set updated = true with ratelimit ( 10 rcu, 5 wcu );
update ddbsh_demo remove c where a = 5; (works like alter table table-name drop column column-name)

UPSERT <name> SET <upsert_set> [where clause] [ratelimit]
upsert ddbsh_demo set op = "upsert" where pk = 2;

Deleting records from DynamoDB tables using DDB shell

delete from ddb_tabble where name = "Brutus";
delete from ddbsh_demo where pk = 1 and rk = "one";

Backup and Restore of AWS DynamoDB tables

backup table dynamodb_tab called dynamodb-table-backup;
show backups;
describe backup "arn:aws:dynamodb:us-east-2:1234567890:table/dynamodb_tab/backup/1681480720-2a11c134";
drop backup "arn:aws:dynamodb:us-west-2:1234567890:table/backup_test/ddbsh_demo/1681480720-2a11c134";

restore table "dynamodb_tab" from backup "arn:aws:dynamodb:us-east-2:1234567890:table/dynamodb_tab/backup/1681480720-2a11c134";
RESTORE TABLE new table name FROM old table name TO PITR "YYYY-MM-DDTHH:MM:SSZ";

Getting EXPLAIN plan for DynamoDB (shell) queries

explain select * from ddbsh_demo2;
explain select * from dynamodb_tab where attribute_type(b, string) or (a = 5 and c = "coffee");
explain create table ddbsh_demo (id string, name string) primary key (id hash) billing mode provisioned (5 rcu, 5 wcu) gsi (namegsi on (name hash) projecting all billing mode provisioned (5 rcu, 5 wcu));
explain alter table ddb_tab add replica us-west-2;
explain alter table ddbsh_demo (v number) create gsi gsi_v on (v hash) projecting all billing mode provisioned (10 rcu, 20 wcu);
explain update ddbsh_demo set z = 14, v.b[6] = 13 where id = 5;
explain update ddb_table.zipgsi set state = "CA" where zip = "90210" with ratelimit (10 rcu, 20 wcu);
explain delete from ddb_tab where rk = "two";


Related AWS Cloud Articles:  AWS Cloud Database/DBA Interview Questions

July 12, 2025

Percona Xtrabackup Backup & Recovery

Percona Xtrabackup (PXC Part 4)


XtraBackup is an open-source MySQL hot backup software program.
Features include hot, non-locking backups for InnoDB storage, incremental backups, streaming, parallel-compressed backups, throttling based on the number of I/O operations per second, etc.

Percona XtraBackup is an open-source hot backup utility for MySQL servers that doesn’t lock databases during the backup.


Percona Xtrabackup (8.0) installation from Repository

Percona provides repositories for yum (RPM packages for Red Hat, CentOS and Amazon Linux AMI) and apt (.deb packages for Ubuntu and Debian) for software such as Percona Server for MySQL, Percona XtraBackup, and Percona Toolkit.

yum list all | grep xtrabackup
dnf install percona-xtrabackup-80 -y

Percona XtraBackup can backup data from InnoDB, XtraDB, MyISAM tables on MySQL 8.0 servers as well as Percona Server for MySQL with XtraDB, Percona Server for MySQL 8.0, and Percona XtraDB Cluster 8.0.

Creating Backup

To create a backup, run xtrabackup with --backup option and specify --target-dir option where the backup will be stored.
xtrabackup --backup --target-dir=/data/backups/
xtrabackup --backup --user=satya --password=Secret9 --target-dir=/path/to/backupdir
xtrabackup --backup --user=satya --password=Secret9 --slave-info --target-dir=/path/to/backupdir

Preparing Backup

After making a backup with the --backup option, you need need to prepare it in order to restore it.
xtrabackup --prepare --target-dir=/data/backups/
xtrabackup --prepare --user=satya --password=Secret9 --target-dir=/path/to/backupdir

Restoring Backup

For convenience, xtrabackup binary has the --copy-back option to copy the backup to the datadir of the server. Backup needs to be prepared before it can be restored.
xtrabackup --copy-back --target-dir=/data/backups/

Incremental Backup

Percona xtrabackup supports incremental backups, which means that they can copy only the data that has changed since the last backup.
DBAs can perform many incremental backups between each full backup, to have a backup process such as a full backup once a week and an incremental backup every day, or full backups every day and incremental backups every hour.

To make an incremental backup, begin with a full backup as usual. The xtrabackup binary writes a file called xtrabackup_checkpoints into the backup’s target directory.
xtrabackup --backup --target-dir=/data/backups/full
xtrabackup --backup --target-dir=/root/backups/full -uroot --pSecret9

Now that you have a full backup, you can make an incremental backup based on it.
xtrabackup --backup --target-dir=/data/backups/incl --incremental-basedir=/data/backups/full
xtrabackup --backup --target-dir=/root/backups/incl --incremental-basedir=/root/backups/full -uroot --pSecret9

Preparing Incremental Backups

The --prepare step for incremental backups is not the same as for full backups. Beginning with the full backup, you can prepare it, and then apply the incremental differences to it.
xtrabackup --prepare --apply-log-only --target-dir=/data/backups/full
xtrabackup --prepare --apply-log-only --target-dir=/root/backups/full -uroot --pSecret9

To apply the first incremental backup to the full backup. Preparing the second/third incremental backup is a similar process.
xtrabackup --prepare --apply-log-only --target-dir=/data/backups/full --incremental-dir=/data/backups/incl
xtrabackup --prepare --apply-log-only --target-dir=/root/backups/full --incremental-dir=/root/backups/incl -uroot --pSecret9

Restoring Incremental Backups

Once prepared incremental backups are the same as the full backups and they can be restored in the same way.

.. shutdown database ..
xtrabackup --copy-back --target-dir=/root/backups/
xtrabackup --copy-back --target-dir=/data/backups/

Related MySQL Articles: Percona Monitoring and Management (PMM)   How to use tools in Percona Toolkit

October 1, 2024

Top New Features in PostgreSQL 16 database

Postgres 16 new features


  • Random connection load balancing support using libpq, which will make it easier to scale out read queries in Postgres version 16.
  • In PostgreSQL 16, we can grant (only) vacuum and analyze operation privileges to another user, no need to be done only the owner or super users.
  • postgres=> grant analyze on table1 to user1;
    postgres=> grant vacuum on table1 to user1;
    postgres=> grant pg_vacuum_all_tables to satya;
    postgres=> grant pg_analyze_all_tables to satya;
  • Users with CREATEROLE privilege, without ADMIN OPTION, no longer have the ability to hand out membership in any role to anyone. 
  • GRANT query to be performed with ADMIN OPTION.
  • The speed of hash index builds improved.
  • Logical replication from standby servers. Logical replication subscribers to apply large transactions in parallel.
  • Regular expression matching on database and role entries in pg_hba.conf.
  • Postgres 16 onwards, NUMERIC can process hexadecimal, octal, and binary integers of any size.
  • PostgreSQL query planner can now parallize FULL and RIGHT joins, by creating gather node and worker nodes.
  • Supports incremental sorts for SELECT DISTINCT.

New System Views in Postgres 16:
  • Monitoring of I/O statistics using the new pg_stat_io view.

New Parameters in PostgreSQL 16:
  • New server variable enable_presorted_aggregate to aggregates having ORDER BY or DISTINCT to use pre-sorted data enable/disable.
  • Added server variable SYSTEM_USER.
  • New server parameter logical_replication_mode to control how logical decoding publishers transfer changes and how subscribers apply them.

Related PostgreSQL Articles: New features of Postgres 15

June 25, 2024

Installation & Usage of Percona Toolkit 3

Percona Toolkit Usage and Installation (PXC Part 3)


Percona Toolkit is a collection of advanced command-line (CLI) tools that are created/used by Percona to help Percona support team or users to perform a variety of MongoDB and MySQL administration tasks that can be too difficult or complex to do manually.


Installing Percona Toolkit (3.3)


Install Percona Toolkit using the corresponding package manager, on RHEL or CentOS:
yum list all | grep toolkit
dnf install percona-toolkit -y

If you want to download a specific tool, use http://www.percona.com/get
For example, to download the pt-slave-restart tool, run:
wget percona.com/get/pt-slave-restart

Using tools in Percona Toolkit

pt-summary # Summarize system information nicely
pt-summary
pt-summary > pt-summary.out

pt-mysql-summary # Summarize MySQL information nicely
pt-mysql-summary --user=root --password=secret
pt-mysql-summary --user=root --password=secret  > "${PTDEST}/pt-mysql-summary.out"

pt-table-checksum    # Checks whether master/replicas have consistent copy or not
pt-table-checksum -uroot --pSecret9
pt-table-checksum -uroot --pSecret9 h=localhost -d thirumani
pt-table-checksum -uroot --pSecret9 h=localhost -d thirumani --slave-user=replication --slave-password=Slave@123
pt-table-checksum -uroot --pSecret9 h=localhost -d thirumani --slave-user=replication --slave-password=Slave@123 --no-check-binlog-format
pt-table-checksum -uroot --pSecret9 --replicate testdb.sample h=master-host
pt-table-checksum -uroot --pSecret9 --replicate testdb.sample h=master-host -d sakila

pt-table-sync # Synchronize MySQL table data efficiently
pt-table-sync --execute h=localhost,D=thirumani,t=order_items_refund h=host2
pt-table-sync --execute h=localhost -uroot --pSecret9 --replicate thirumani.order_items_refund --slave-user=replication --slave-password=Slave@123
pt-table-sync --execute localhost host2 host3
pt-table-sync --execute --sync-to-master slave1
pt-table-sync --execute --replicate test.checksum master1
pt-table-sync --execute --replicate test.checksum --sync-to-master slave1
pt-table-sync --execute --sync-to-master h=master2,D=db,t=tbl
pt-table-sync -uroot --pSecret9 --replicate testdb.sample --print h=master-host

pt-slave-find # Find and print replication hierarchy tree of MySQL slaves
pt-slave-find --host localhost -uroot --pSecret9

pt-slave-delay # Make a MySQL slave server lag behind its master
pt-slave-delay --delay 1m --interval 15s --run-time 10m slavehost
pt-slave-delay --delay 2h h=slave

pt-slave-restart # Watch and restart MySQL replication after errors
pt-slave-restart [OPTIONS] [DSN]
pt-slave-restart
pt-slave-restart h=slave
pt-slave-restart --master-uuid

pt-heartbeat # Monitor MySQL replication delay
pt-heartbeat -h192.168.1.222 -uroot --pSecret9
pt-heartbeat -D percona --create-table --update h=master
pt-heartbeat -D test --update -h master-server --daemonize
pt-heartbeat -D test --monitor -h slave-server
pt-heartbeat -D test --check h=slave-server

pt-query-digest # Analyze MySQL queries from logs, processlist, and tcpdump
pt-query-digest master/data/slow.log
pt-query-digest --review h=host2 --no-report slow.log
pt-query-digest slow.log --no-report --output slowlog --filter '$event->{fingerprint} && make_checksum($event->{fingerprint}) eq "FDEA8D2993C9CAF3"'
pt-query-digest --create-review_table --review D=dbname,t=tablename slow.log
pt-query-digest --processlist h=localhost -uroot --pSecret9
pt-query-digest --processlist h=localhost -uroot --pSecret9 --print --no-report --interval=0.01 > slow.log
pt-query-digest --processlist h=localhost -uroot --pSecret9 --interval=0.01 --output=slowlog > slow.log
tcpdump -s 65535 -x -nn -q -tttt -i any -c 1000 port 3306 > mysql.tcp.txt
pt-query-digest --type tcpdump mysql.tcp.txt

pt-upgrade # Verify that query results are identical on different servers (before upgrade)
pt-upgrade h=host1 h=host2 slow.log
pt-upgrade h=host1 --save-results host1_results/ slow.log
pt-upgrade host1_results1/ h=host2
pt-upgrade h=host1 h=host2 queries.txt

pt-visual-explain # Format EXPLAIN output as a tree
pt-visual-explain <file_containing_explain_output>
pt-visual-explain -c <file_containing_query>
mysql -e "explain select * from mysql.user" | pt-visual-explain

pt-stalk # Collect forensic data about MySQL when problems occur
pt-stalk --function process list --variable Command --match Sleep --threshold 155 --cycles 0
pt-stalk --no-stalk --iterations=2 --sleep=30 -- --user=root --password=<mysql-root-pass>
pt-stalk --no-stalk --iterations=2 --sleep=30 --dest="${PTDEST}" -- --user=root --password=<mysql-root-pass>
pt-stalk --variable Threads_connected --cycles 2 -- --user satya

pt-find #  Find MySQL tables and execute actions, like GNU find
pt-find --ctime +1 --engine MyISAM
pt-find --engine InnoDB --exec "ALTER TABLE %D.%N ENGINE=MyISAM"
pt-find --connection-id '\D_\d+_(\d+)$' --server-id '\D_(\d+)_\d+$' --exec-plus "DROP TABLE %s"
pt-find --empty junk test --exec-plus "DROP TABLE %s"
pt-find --tablesize +5G
pt-find --printf "%T\t%D.%N\n" | sort -rn
pt-find --noquote --exec "INSERT INTO sysdata.tblsize(db, tbl, size) VALUES('%D', '%N', %T)"

====================================================

pt-align # Aligns output from other tools to columns

pt-archiver # Archive rows from a MySQL table into another table or a file
pt-archiver --source h=host,D=db,t=child --purge --where 'NOT EXISTS(SELECT * FROM parent WHERE col=child.col)'
pt-archiver --source h=oltp_server,D=test,t=tbl --dest h=olap_server --file '/var/log/archive/%Y-%m-%d-%D.%t' --where "1=1" --limit 1000 --commit-each
pt-archiver --source h=127.0.0.1,D=sakila,t=payment --where 'rental_id > 1000' --limit 100 --commit-each --purge --dry-run
pt-archiver --source h=127.0.0.1,D=sakila,t=payment --where 'rental_id > 1000' --limit 100 --commit-each --purge --progress=100
pt-archiver --source h=127.0.0.1,D=sakila,t=payment --dest h=12.0.0.1,D=sakila_archive,t=payment --where 'rental_id > 1000' --limit 100 --commit-each

pt-config-diff # Diff MySQL configuration files and server variables
pt-config-diff h=host1 h=host2
pt-config-diff /etc/my.cnf h=host1
pt-config-diff /etc/my-small.cnf /etc/my-large.cnf

pt-deadlock-logger # Log MySQL deadlocks
pt-deadlock-logger h=host1
pt-deadlock-logger h=host1 --iterations 1
pt-deadlock-logger h=host1 --dest h=host2,D=percona_schema,t=deadlocks

pt-diskstats # An interactive I/O monitoring tool for GNU/Linux
pt-diskstats [OPTIONS] [FILES]
pt-diskstats

pt-duplicate-key-checker # Find duplicate indexes and foreign keys on MySQL tables
pt-duplicate-key-checker --host host1

pt-fifo-split # Split files and pipe lines to a fifo without really splitting
pt-fifo-split --lines 1000000 hugefile.txt
while [ -e /tmp/pt-fifo-split ]; do cat /tmp/pt-fifo-split; done

pt-fingerprint # Convert queries into fingerprints
pt-fingerprint --query "select a, b, c from users where id = 500"
pt-fingerprint /path/to/file.txt

pt-fk-error-logger # Log MySQL foreign key errors
pt-fk-error-logger h=host1
pt-fk-error-logger h=host1 --iterations 1
pt-fk-error-logger h=host1 --dest h=host2,D=percona_schema,t=fke

pt-index-usage # Read queries from a log and analyze how they use indexes
pt-index-usage /path/to/slow.log --host localhost
pt-index-usage slow.log --no-report --save-results-database percona

pt-ioprofile # Watch process IO and print a table of file and I/O activity
pt-ioprofile [OPTIONS] [FILE]
pt-ioprofile

pt-kill # Kill MySQL queries that match certain criteria
pt-kill --busy-time 60 --kill
pt-kill --busy-time 60 --print
pt-kill --match-command Sleep --kill --victims all --interval 10
pt-kill --match-state login --print --victims all
mysql -e "SHOW PROCESSLIST" > proclist.txt
pt-kill --test-matching proclist.txt --busy-time 60 --print

pt-mext # Look at many samples of MySQL SHOW GLOBAL STATUS side-by-side
pt-mext -r -- mysqladmin ext -i10 -c3
pt-mext -r -- cat mysqladmin-output.txt
pt-mext -r -- mysqladmin ext -i1 -c4
pt-mext -r -- cat 2021_05_01_12_55_30-mysqladmin > ptmext.out

pt-online-schema-change # ALTER tables without locking them
pt-online-schema-change --alter "ADD COLUMN c1 INT" D=sakila,t=actor
pt-online-schema-change --alter "ENGINE=InnoDB" D=sakila,t=actor
pt-online-schema-change --alter-foreign-keys-method auto --alter "add key actor_last_update(last_update)" --execute h=localhost,D=sakila,t=actor

pt-pmp # Aggregate GDB stack traces for a selected program, a poor man’s profiler
pt-pmp [OPTIONS] [FILES]

pt-secure-collect # collect, sanitize, pack and encrypt data
pt-secure-collect [<flags>] <command> [<args> ...]
pt-secure-collect collect <flags>

pt-show-grants #Canonicalize and print MySQL grants so you can effectively replicate, compare and version-control them
pt-show-grants
pt-show-grants --separate --revoke | diff othergrants.sql -

pt-sift # Browses files created by pt-stalk
pt-sift FILE|PREFIX|DIRECTORY

pt-table-usage # Analyze how queries use tables
pt-table-usage [OPTIONS] [FILES]

pt-variable-advisor # Analyze MySQL variables and advise on possible problems
pt-variable-advisor localhost
pt-variable-advisor --source-of-variables vars.txt

pt-mongodb-query-digest # reports query usage statistics by aggregating queries from MongoDB query profiler
pt-mongodb-query-digest [OPTIONS]
pt-mongodb-summary # collects information about a MongoDB cluster. It collects information from several sources to provide an overview of the cluster
pt-mongodb-summary [OPTIONS] [HOST[:PORT]]

pt-pg-summary # collects information about a PostgreSQL cluster
pt-pg-summary [options] [host:[port]]

pt-k8s-debug-collector # Collects debug data (logs, resource statuses etc.) from a k8s/OpenShift cluster. Data is packed into the cluster-dump.tar.gz archive in the current working directory
pt-k8s-debug-collector

Related Percona/MySQL Articles: Percona XtraDB Cluster (PXC) Configuration   Percona Xtrabackup Full/Incremental Backup & Recovery

February 10, 2024

50 Top OGG (Oracle GoldenGate) Interview Questions

Oracle Golden Gate (OGG) Interview Questions/answers Part 1


1. What is GoldenGate and how to set up Golden Gate?

Answer:

2. What are the processes/components in OGG (Oracle Golden Gate)?

Answer:
Manager, Extract, Replicat, Data Pump

3. What is the use of (Data) Pump process in Golden Gate (GG)?
Answer:

4. What is the command-line utility in Oracle GoldenGate (or) what is ggsci?

Answer:

5. What is the default port for Golden Gate Manager process?

Answer:
7809

6. What are the important files of Oracle Golden Gate (OGG)?

Answer:
GLOBALS, ggserr.log, dirprm, etc ...

7. What is a checkpoint table?

Answer:

8. How can you see Golden Gate errors?

Answer:
ggsci> VIEW GGSEVT
ggserr.log file

9. What is the difference between classical extract and integrated extract?

Answer:
Classic Capture/extract:      (deprecated in OGG 21c)
* The Classic Capture mode is the traditional Extract process that accesses the database redo logs (optionally archive logs) to capture the DML changes occurring on the objects specified in the parameter files.
* At the OS level, the GoldenGate user must be a part of the same database group which owns the database redo logs.
* This capture mode is available for other RDBMS as well.
* There are some data types that are not supported in Classic Capture mode.
* Classic capture can’t read data from the compressed tables/tablespaces.

Integrated Capture (IC)/Extract:
* In the Integrated Capture/Extract mode, GoldenGate works directly with the database log mining server to receive the data changes in the form of logical change records (LCRs).
* IC mode does not require any special setup for the databases using ASM, transparent data encryption, or Oracle RAC.
* This feature is only available for oracle databases in Version 11.2.0.3 or higher.
*  It also supports various object types which were previously not supported by Classic Capture.
* This Capture mode supports extracting data from source databases using compression.
* Integrated Capture can be configured in an online or downstream mode.

10. What is the difference between classical replicat and integrated replicat?
Answer:

11. What is the difference between classical replicat and coordinated replicat?
Answer:

12. What is the use assumetargetdefs in replicat parameter file?
Answer:
Use the ASSUMETARGETDEFS parameter when the source and target objects specified in a MAP statement have identical column structure, such as when synchronizing a hot site. It directs Oracle GoldenGate to assume that the data definitions of the source and target objects are identical, and to refer to the target definitions when metadata is needed for the source data.

13. Can you configure multiple extracts to write to the same exttrail file?
Answer:
Only one Extract process can write to one exttrail at a time. So you can’t configure multiple extracts to write to the same exttrail.

14. What is the use of PASSTHRU and NOPASSTHRU parameters ?
Answer:
parameter passthru indicates the source and target database tables structure are identical. In pass-through mode (configured by PASSTHRU), the Extract process does not lookup table definitions from the database or from a data-definitions file. To use pass-through mode, the source and target table names and structures must be identical.

15. What is a Handle collision in OGG?
Answer:
The Golden Gate HANDLECOLLISIONS parameter is configured on the target database in the Replicat process to enable processing of the data when there are duplicate data integrity issues in the destination database.
Use the HANDLECOLLISIONS and NOHANDLECOLLISIONS parameters to control whether or not Replicat tries to resolve duplicate-record and missing-record errors when applying SQL on the target.

16. List important considerations for bi-directional (active-active setup) replication?
Answer:
The customer should consider the following points in an active-active replication environment.
* Primary Key: Helps to identify conflicts and resolve them.
* Sequences: Are not supported. The workaround is used to use odd/even, range or concatenate sequences.
* Triggers: These should be disabled or suppressed to avoid using uniqueness issue
* Data Looping: This can easily be avoided using OGG itself
* LAG: This should be minimized. If a customer says that there will not be any LAG due to network or huge load, then we don’t need to deploy CDR. But this is not the case always as there would be some LAG and these can cause Conflicts.
* CDR (Conflict Detection & Resolution): OGG has built-in CDRs for all kinds of DMLs that can be used to detect and resolve them.
* Packaged Application: These are not supported as they may contain data types that are not supported by OGG or it might not allow the application modification to work with OGG.

17. What is the difference between Discard and Ignore (in RESOLVECONFLICT) ?
Answer:

18. How you will check whether replicat is classical replicat or coordinated replicat or integrated replicat?
Answer:

19. How to find out the RBA of a Golden Gate (OGG) process ? [or] Which command will you use to find out the RBA of extract/replicat ?
Answer:

20. What is the heartbeat table in OGG ? Who will insert data into the heartbeat table ?
Answer:

21. How to check whether the table is in OGG replication or not ?
Answer:

22. What is trace table? What is the use of trace table?
Answer:

23. What is the use of initial load ?
Answer:

24. 
What are the different OGG Initial load methods available?
Answer:

25. 
Answer:



Related Oracle Articles: Oracle DBA Interview Questions/FAQs – Part3   Oracle Performance related interview Questions/FAQs

October 1, 2023

30 Top Oracle RMAN basic/advanced Interview Questions for DBAs

Frequently asked Oracle DBA RMAN (Recovery Manager) Real-time Interview Questions


0. What is RMAN and why to use RMAN?
Answer :

1. Difference between catalog and nocatalog?
Answer :
Recovery catalog is central and can have information from many databases.

2. Difference between using the recovery catalog and the control file in Oracle?
Answer :
When a new incarnation happens, the old backup information in the control file will be lost. It will be preserved in the recovery catalog.
In the recovery catalog, we can store scripts.
Recovery catalog is central and can have information from many databases.

3. Can we use the same target database as catalog?
Answer :
No. The recovery catalog should not reside in the target database (database to be backed up), because the database can't be recovered in the mounted state.

4. As DBA, how do you know the progress of the Oracle RMAN task?
Answer :
By querying v$rman_status or v$session_longops

5. From where list and report commands will get input?
Answer :
Both the commands command querying v$ and recovery catalog (RC) views. V$BACKUP_FILES or many of the recovery catalog views such as RC_DATAFILE_COPY or RC_ARCHIVED_LOG.

6. What is the command to delete archive logs older than 7days?
Answer :
RMAN> delete archivelog all completed before sysdate-7;

7. By default how many days backup RMAN stores in Oracle Database?
Answer :

8. What is the use of crosscheck command in RMAN?
Answer :
Crosscheck will be useful to check whether the catalog information is intact with OS-level information.

9. What are the differences between crosscheck and validate commands?
Answer :
Validate command is to examine a backup set and report whether it can be restored. RMAN scans all of the backup pieces in the specified backup sets and looks at the checksum to verify that the contents are intact so that the backup can be successfully restored if necessary.

Crosscheck command is to verify the status of backups and copies recorded in the RMAN repository against media such as disk or tape. The crosscheck command only processes files created on the same device type as the channel running crosscheck.

10. Which is one is good, differential (incremental) backup or cumulative (incremental) backup?
Answer :
A differential backup, which backs up all blocks changed after the most recent incremental backup at level 1 or 0.
A cumulative backup, which backs up all blocks changed after the most recent incremental backup at level 0.

11. What are Level 0, Level 1 backups in Oracle?
Answer :
A level 0 incremental backup, which is the base for subsequent incremental backups, copies all blocks containing data, backing the datafile up into a backup set just as a full backup would. A level 1 incremental backup can be either of the following types:
A differential backup, which backs up all blocks changed after the most recent incremental backup at level 1 or 0.
A cumulative backup, which backs up all blocks changed after the most recent incremental backup at level 0.

12. Can we perform level 1 backup without level 0 backup?
Answer :
If no level 0 backup is available, then the behavior depends upon the compatibility mode setting.
If compatibility < 10.0.0, RMAN generates a level 0 backup of the file contents at the time of the backup.
If compatibility is >= 10.0.0, RMAN copies all blocks changed since the file was created, and stores the results as a level 1 backup. In other words, the SCN at the time the incremental backup is taken is the file creation SCN.

13. Will RMAN put the database/tablespace/datafile in backup mode?
Answer :
Nope.

14. What is a snapshot control file in Oracle?
Answer :
The snapshot control file is a copy of a database control file created in an operating system-specific location by RMAN. RMAN creates the snapshot control file so that it has a consistent version of a control file to use when either resynchronizing the recovery catalog or backing up the control file.

15. What is the difference between a backup set and a backup piece?
Answer :
Backup set is logical and backup piece is physical.

16. RMAN command to backup for creating standby database?
Answer :
RMAN> duplicate target database to standby database ....

17. How to do cloning by using Oracle RMAN?
Answer :
RMAN> duplicate target database …

18. Suppose you lost one datafile and DB is running in ARCHIVELOG mode. You have full database backup of 1 week/day old and don’t have a backup of the (newly created) datafile. How do you restore/recover file?
Answer :
Create the datafile and recover that datafile.
SQL> alter database create datafile '…path...' size n;
RMAN> recover datafile file_id;


19. What is an obsolete backup and expired backup?
Answer :
A status of "expired" means that the backup piece or backup set is not found in the backup destination.
A status of "obsolete" means the backup piece is still available, but it is no longer needed. The backup piece is no longer needed since RMAN has been configured to no longer need this piece after so many days have elapsed, or so many backups have been performed.

20. What is the difference between a hot backup and RMAN backup?
Answer :
For hot backup, we have to put the database in begin backup mode, then take backup.
RMAN won’t put the database in backup mode.

21. How to put manual/user-managed backup in RMAN (recovery catalog)?
Answer :
By using catalog command.
RMAN> CATALOG START WITH '/tmp/backup.ctl';

22. What were the new features in Oracle 11g RMAN?
Answer :

23. What were the new features in Oracle 12c RMAN?
Answer :

24. What is the difference between the auxiliary channel and the maintenance channel?
Answer :

25. What is the use of nofilenamecheck in RMAN ?
Answer :
When NOFILENAMECHECK is used with the DUPLICATE command, RMAN does not validate the filenames during restoration. If the primary database and the standby database are on the same host, The DUPLICATE NOFILENAMECHECK option should not be used.

26. Do we need to shutdown the database to change the block change tracking file?
Answer :
No need

27. How will you identify a Physical Corruption/Logical corruption?
Answer :

28. What happens if you lose SPFILE and PFILE and DB crashed with some error, how will you start the database?
Answer :

29. What are the database file's that RMAN cannot backup ?
Answer :
RMAN cannot take backup of the pfile, redo logs, TNS or network configuration files, password files, external tables and the contents of the Oracle home files.

30. Can I take RMAN backup when the database is down ?
Answer :
No, we can take RMAN backup only when the target database is open or in the mount stage.

31. How to check RMAN configuration ?
Answer :

32. Can we have archive logs and datafile backup in a single backupset ?
Answer : No

33. Can we take RMAN backup when the target database is down ?
Answer : No

Related Oracle interview Articles: MySQL DBA Interview Questions and Answers   Oracle PL/SQL Interview Questions/FAQs   Oracle Export/Import - Data Pump Interview Questions/FAQs


September 2, 2023

Top New Features in MySQL 8.1 database

MySQL 8.1 new features


Here are the top new features in 8.1.0 release of MySQL database version:

  • Logging of the shutdown process has been enhanced, with the addition of startup and shutdown messages for the MySQL server, plugins, and components.
  • Release MySQL 8.1 extends EXPLAIN FORMAT=JSON with an INTO a user variable where it can be worked with using MySQL JSON functions.
  • Added the SHOW PARSE_TREE statement, which shows the JSON-formatted parse tree for a SELECT statement.
    SHOW PARSE_TREE SELECT * FROM t3 WHERE id > 2\G
  • Stripping of comments from the mysql client was the default behavior; from MySQL 8.1.0 onwards, the default has changed to preserving such comments.
  • Default value for the SOURCE_RETRY_COUNT option of the CHANGE REPLICATION SOURCE TO statement has been changed to 10.
  • Added the tls-certificates-enforced-validation system variable, which permits a DBA to enforce certificate validation at server startup or when using the ALTER INSTANCE RELOAD TLS statement to reload certificates at runtime.
  • group_replication_set_as_primary() function now waits for ongoing DDL statements such as ALTER TABLE when waiting for all transactions to complete, prior to electing a new primary.
  • Added server system variables to control the amount of time MySQL accounts that connect to a MySQL server using LDAP pluggable authentication must wait when the LDAP server is down or unresponsive.
  • Added status variables specific to Group Replication plugin that improve diagnosis and troubleshooting of network instabilities, providing statistics about network usage, control messages, and data messages for each group member.

Related MySQL Articles:  MySQL DBA top basic/advanced interview questions - Part 1

August 4, 2023

Top 50 MySQL DBA interview questions answers

MySQL/MariaDB DBA Interview Questions and Answers - Part 1


Q1. What is the MySQL default port number?
answer: 3306

Q2. How MySQL replication works?
answer:
Replication is the process of creating and maintaining multiple copies of a MySQL database. It allows you to synchronize data across multiple database servers, providing high availability, scalability, and redundancy.

Q3. How will you fix MySQL slave replication issues?
answer:

Q4. What is GTID based replication?
answer:

Q5. Please explain the main differences between RDS MySQL and Amazon Aurora MySQL cluster?
answer:

Q6. Define ACID properties?
answer:

Q7. How many types of backups are available in MySQL? How do you take a backup of a MySQL database?
answer:
MySQL provides multiple methods for backup, such as using the mysqldump utility, using the MySQL Enterprise Backup tool, or performing raw data backups using file system-level tools.

Q8. How do you take incremental backup in MySQL?
answer: 

Q9. Give the different types of logs available in MySQL?
answer:

Q10. What is the command to retrieve the current date in MySQL?
answer:

Q11. Which is the default database engine utilized in MySQL?
answer:

Q12. Provide the various database engines present in MySQL?
answer:

Q13. How can a user get the current MySQL version?
answer:

Q14. What is Sharding in MySQL?
answer:

Q15. Is there any way to optimize MySQL tables? What is optimize table command do?
answer:

Q16. What are the Transaction Isolation Levels in MySQL database?
answer:

Q17. What is the default Transaction Isolation Level in MySQL?
answer:

Q18. How to change Transaction Isolation Level in MySQL?
answer:

Q19. What is MVCC in MySQL database?
answer:

Q20. How to improve the performance of a MySQL query?
answer:

    
Related MySQL Articles: 50 
Best MySQL DBA interview questions answers Part 2   MySQL Database Administrator Online Quiz

July 23, 2023

Frequently asked Oracle PL/SQL Interview Questions

Oracle PLSQL Interview Questions/answers


1. What is the use of package/function/procedure?
Answer:
A Procedure that performs an action and a function that computes a value.
Package is a collection of related variables, cursor, procedure and functions. A package is a schema object which groups logically related PL/SQL types, items and subprograms.
Functions return a single value, procedures are used to return multiple values, otherwise, it is generally similar to a function. 
Functions can be used in SQL query but procedure we can't use in SQL query. 

2. What is the use of FORALL, BULK COLLECT in PL/SQL?
Answer:

3. What is PL/SQL table (now PL/SQL tables are called Associative Arrays)?
Answer:
PL/SQL table or associative array is a one-dimensional array.
An associative array is an unbounded set of key-value pairs. Each key is unique, and serves as the subscript of the element that holds the corresponding value.

4. How to compile procedure/package?
Answer:
alter procedure upd_int compile;
alter package SCOTT.LOCA_SER compile;

5. What are the differences between if and nullif?
Answer:

6. What is the maximum number of triggers, can apply to a single table?
Answer:
12 triggers

7. What are the differences between decode and case?
Answer:

8. What is pragma?
Answer:
Triggers can run under PRAGMA AUTONOMOUS_TRANSACTION, in which case they must have a COMMIT. But this is an edge case, as there are few meaningful uses of nested transactions in Oracle.
Pragma is a compiler directive in oracle.
Autonomous_Transaction is creating a new or independent session in the same transaction.
Autonomous Transaction provides functionality to the developer in which it allows to do changes in a separate transaction and to save/discard that particular transaction without affecting the main session transaction.

9. Want output in below format, salary in descending order and with serial numbers.
Answer:
Empno empname   Sal S.No
123     Satya      12000 1
345     Rajesh     10000 2
796     Prasad      9000 3
986     Bhanu       8000 4

select t1.*, rownum from (select empno, empname, sal from emp order by sal desc)t1 order by rownum;


10. What is the difference between function and procedure?
Answer:
Functions return a single value, procedures are used to return multiple values, otherwise, it is generally similar to a function. 
Functions can be used in SQL query but procedure we can't use in SQL query. 
A Procedure that performs an action and a function that computes a value.
Package is a collection of related variables, cursor, procedure and functions. A package is a schema object which groups logically related PL/SQL types, items and subprograms.

11. What is mutating of a trigger?
Answer:
A mutating table error (ORA-04091) occurs when a row-level trigger tries to examine or change a table that is already undergoing change (via an INSERT, UPDATE, or DELETE statement). In particular, this error occurs when a row-level trigger attempts to read or write the table from which the trigger was fired.

12. What are cursor attributes?
Answer:
Cursor attributes are used to get information about the current status of the cursor.

Both explicit and implicit cursors have four attributes:

• FOUND – It Returns TRUE if the record was fetched successfully, FALSE otherwise
• NOTFOUND - Returns TRUE if the record was not fetched successfully, FALSE otherwise.
• ROWCOUNT - Returns the number of records fetched from the cursor at that point in time.
• ISOPEN - Returns TRUE if the cursor is open, FALSE otherwise.

13. What are the different types of triggers?
Answer:

14. Can trigger contain commit?
Answer:
No, regular triggers can not contain commit. 
Any change that a trigger does is committed with the transaction that fired the trigger.
You will get the below error.
ORA-04092: cannot COMMIT in a trigger

Autonomous triggers (PRAGMA AUTONOMOUS_TRANSACTION) can contain COMMIT and ROLLBACK.

15. Can you call a procedure/function inside a trigger?
Answer:
Yes, you can fire a procedure/function from a Trigger.
But, keep in mind that trigger & procedure/function should not access the same table.

Related Oracle Articles: Oracle Exadata Interview Questions   Oracle Data Guard Interview Questions/FAQs  Oracle Export/Import - Data Pump Interview Questions/FAQs