Tuesday, 23 August 2016

LATCH

What Are Latches? 

Latches are serialization mechanisms that protect areas of Oracle’s shared memory (the SGA). In simple terms, latches prevent two processes from simultaneously updating and possibly corrupting the same area of the SGA. 
A latch is a type of a lock that can be very quickly acquired and freed. Latches are typically used to prevent more than one process from executing the same piece of code at a given time.

Oracle sessions need to update or read from the SGA for almost all database operations. For nstance:

• When a session reads a block from disk, it must modify a free block in the buffer cache and adjust the buffer cache LRU chain
• When a session reads a block from the SGA, it will modify the LRU chain.
• When a new SQL statement is parsed, it will be added to the library cache within the SGA.
• As modifications are made to blocks, entries are placed in the redo buffer.
• The database writer periodically writes buffers from the cache to disk (and must update their status from dirty to clean).
• The redo log writer writes entries from the redo buffer to the redo logs. Latches prevent any of these operations from colliding and possibly corrupting the SGA.


Latches vs Enqueues 

Enqueues are another type of locking mechanism used in Oracle. An enqueue is a more sophisticated mechanism which permits several concurrent processes to have varying degree of sharing of "known" resources.
 Any object which can be concurrently used, can be protected with enqueues. A good example is of locks on tables.
We allow varying levels of sharing on tables e.g. two processes can lock a table in share mode or in share update mode etc. One difference is that the enqueue is obtained using an OS specific locking mechanism.
An enqueue allows the user to store a value in the lock, i.e the mode in which we are requesting it. 
The OS lock manager keeps track of the resources locked. If a process cannot be granted the lock because it is incompatible with the mode requested and the lock is requested with wait, the OS puts the requesting process on a wait queue which is serviced in FIFO. Another difference between latches and enqueues is that in latches there is no ordered queue of waiters like in enqueues. Latch waiters may either use timers to wakeup and retry or spin (only in multiprocessors). Since all waiters are concurrently retrying (depending on the scheduler), anyone might get the latch and conceivably the first one to try might be the last one to get


How Latches Work 

Because the duration of operations against memory is very small (typically in the order of nanoseconds) and the frequency of latch requests very high, the latching mechanism needs to be very lightweight. If the latch is already in use, Oracle can assume that it will not be in use for long, so rather than go into a passive wait (e.g., relinquish the CPU and go to sleep) Oracle will retry the operation a number of times before giving up and going to passive wait. This algorithm is called acquiring a spinlock and the number of spins before sleeping is controlled by the Oracle initialization parameter _spin_count. The first time the session fails to acquire the latch by spinning, it will attempt to awaken after 10 milliseconds. Subsequent waits will increase in duration and in extreme circumstances may exceed one second. In a system suffering from intense contention for latches, these waits will have a severe impact on response time and throughput.


Causes of contention for specific latches

If a required latch is busy, the process requesting it spins, tries again and if still not available, spins again. The loop is repeated up to a maximum number of times determined by the initialization parameter _SPIN_COUNT. If after this entire loop, the latch is still not available, the process must yield the CPU and go to sleep. Initially is sleeps for one centisecond. This time is doubled in every subsequent sleep. This causes a slowdown to occur and results in additional CPU usage, until a latch is available. The CPU usage is a consequence of the "spinning" of the process. "Spinning" means that the process continues to look for the availability of the latch after certain intervals of time, during which it sleeps.


Latch statistics in AWR report contain following sections

Firstly Look at on AWR TOP wait event  then take a action behalf of latches.

1. Latch Activity
2. Latch Sleep Breakdown
3. Latch Miss Sources
4. Parent Latch Statistics
5. Child Latch Statistics

----------------------------------------------------------------------------------------
Reducing Contention for internal latches
------------------------------------------------------------------

1. Buffer Cache Latches (Cache Buffer Chain Latch Family)

 *. Cache buffer chain latch     
                        - Reduce hot blocks (reorganize table or index / delete and reinsert the data into a table).
             - Before incrementing the parameter DB_BLOCK_BUFFERS check that specific blocks are not causing the contention avoiding memory wasting.
             - Avoid order clause
             - Increase sequence cache size if use this.
             - Poor Disk I/O.
                          - Multiple Buffer pools and adjusting the parameter DB_BLOCK_LRU_LATCHES to have multiple LRU latches will help on reducing latch contention.
             - Lots of physical and logical read.
             - Large full table scan & large index range scan.
             - Unslective indexes.
                         - Increase the size of buffer cache.


---------------------------------------------------------------------------------------------------------------------
Identify the HOT Block / Detect Cache Buffer Chain Wait 
Oracle Metalink Note # 163424.1
----------------------------------------------------------------------------------------

select count(*) child_count, sum(gets) sum_gets, sum(misses) sum_misses, sum(sleeps) sum_sleeps from v$latch_children where name = 'cache buffers chains';


@ this script to locate a hot block:

select /*+ RULE */ e.owner ||'.'|| e.segment_name segment_name, e.extent_id extent#, x.dbablk - e.block_id + 1 block#, x.tch, l.child# from sys.v$latch_children l, sys.x$bh x, sys.dba_extents e
where x.hladdr = 'ADDR' and e.file_id = x.file# and x.hladdr = l.addr and x.dbablk between e.block_id and e.block_id + e.blocks -1 order by x.tch desc;

-------------------------------------
Reduce Cache Buffer Chain
-------------------------------------
suggests using the _db_block_hash_buckets and _db_block_hash_latches undocumented parameters have been suggested as a remedy,
but always check with MOSC before using any undocumented parameters.


Depending on the TCH column (The number of times the block is hit by a SQL statement), you can identify a hotblock. The higher the value of the TCH column, the more frequent the block is accessed by

SQL statements. In order to reduce contention for this object the following mechanisms can be put in place:

1) Examine the application to see if the execution of certain DML and SELECT statements can be reorganized to eliminate contention on the object.
2) Decrease the buffer cache -although this may only help in a small amount of cases.
3) DBWR throughput may have a factor in this as well. If using multiple DBWR's then increase the number of DBWR's
4) Increase the PCTUSED / PCTFREE for the table storage parameters via ALTER TABLE or rebuild. This will result in less rows per block.
5) Consider implementing reverse key indexes (if range scans aren't commonly used against the segment)

 *. Redo allocation latch 
                  - Contention for this latch in Oracle7 can be reduced by decreasing the value of LOG_SMALL_ENTRY_MAX_SIZE 
                                           on multi-cpu systems to force the use of the redo copy latch.
                              - In Oracle8i this parameter is obsolete, so you need to consider to increase the size of the LOG_BUFFER or
                                           reduce the load of the log buffer using NOLOGGING features when possible.

 *. Redo copy latch 
                              - This latch is waited for on both single and multi-cpu systems. On multi-cpu systems, contention can be reduced 
                            by increasing the value of LOG_SIMULTANEOUS_COPIES (Hidden in Oracle8i) and/or increasing 
                        LOG_ENTRY_PREBUILD_THRESHOLD (undocumented in Oracle7).
                 - Increase the size of LOG_BUFFER.

 *. Library cache latch
                    - Ensure that the application is reusing as much as possible SQL statement.
                    - If the application is already tuned, increase the SHARED_POOL_SIZE.

 *. Library cache pin latch
                    - This latch is acquired when a statement in the library cache is reexecuted

  *. Shared pool latches
                               -  Specify the parameteres to reduce it CURSOR_SPACE_FOR_TIME, CURSOR_SHARING=FORCE  
                                    Use Bind variables instead of Literals.
                   - Ways to reduce the shared pool latch are, avoid hard parses when possible.
                              - In order to reduce contention for this latch, we need to tune the data dictionary cache.
                                          In Oracle7 this basically means increasing the size of the shared pool (SHARED_POOL_SIZE) 
                               as the dictionary cache is a part of the shared pool.




References:

Monday, 8 August 2016

RECYCLEBIN WITH RETENTION


Following is a SQL*Plus script which can purge recycle bin with retention date

Syntax: sqlplus system/password@TNS @purge_recyclebin <Days to keep>

REM Script: purge_recyclebin.sql
REM User must has satisfy any of the following priviledge
REM 1. Has DROP ANY priviledge, like SYSTEM, or DBA
REM 2. Or SYSDBA, like SYS

set pagesize 0 feedback off
set verify off

REM Initialize Date format
alter session set nls_date_format = 'DD-MON-YY hh24:mi:ss';

select sysdate || ' Recycle Bin Purging Start' from dual;

col param1 new_value param1

REM prompt Getting parameter 1 from command line
select &1 Param1 from dual;

REM Printing parameter 1 captured
select 'Retention Day: &param1' Param_In from dual;

col type format a8
col ORIGINAL_NAME format a15
col owner format a9
col ts_name format a10

set pagesize 100
prompt
prompt Recyclebin Summary - Before
select substr(droptime, 1, 7) month, owner, count(*) num_obj,
   sum(space) total_space
from dba_recyclebin group by owner, substr(droptime, 1, 7);

prompt
prompt Recyclebin To Be Purge
select substr(droptime, 1, 7) month, owner, type, count(*) num_obj,
   sum(space) total_space
from dba_recyclebin
where droptime < to_char(sysdate - &&param1 , 'yyyy-mm-dd')
group by owner, substr(droptime, 1, 7), type;


REM List of tables in recyclebin
REM select owner, original_name, droptime, TYPE, ts_name, space
REM from dba_recyclebin order by droptime;

set termout off pagesize 0

spool purge_this
select 'purge table ' || owner || '.' || original_name || ';' Purge_command
from dba_recyclebin
where droptime < to_char(sysdate - &&param1 , 'yyyy-mm-dd')
and type = 'TABLE';
spool off

REM Executing the script
set echo on time on
spool purge_this.out
@purge_this.lst
spool off
set echo off

set pagesize 100 termout on

prompt
prompt Recyclebin Summary - After
select substr(droptime, 1, 7) month, owner, count(*) num_obj,
   sum(space) total_space
from dba_recyclebin group by owner, substr(droptime, 1, 7);

prompt

set pagesize 0
select sysdate || ' Recycle Bin Purging End' from dual;
set pagesize 100 termout on verify on feedback on





Question(1) : what’s the recyclebin Retention Period of time and in which parameter they have to work on it  ( undo_retention , db_flashback_retention_target ).?

Ans. 




Question(2): Location of allocate the Recyclebin space. ?

Ans. On tablespace level. When a tablespace is completely filled up with recycle bin data such that the datafiles have to extend to make room for more data, the tablespace is said to be under "space pressure." In that scenario, objects are automatically purged from the recycle bin in a first-in-first-out (FIFO) manner. The dependent objects (such as indexes) are removed before a table is removed.

Similarly, space pressure can occur with user quotas as defined for a particular tablespace. The tablespace may have enough free space, but the user may be running out of his or her allotted portion of it. In such situations, Oracle automatically purges objects belonging to that user in that tablespace.



Reference :

Enable / Disable / Purge Recyclebin

Thursday, 4 August 2016

AWR / ASH Reports

Automatic Workload Repository (AWR) collects performance statistics like Wait events used to identify performance problems, Active Session History (ASH) statistics, some system and session statistics, Object usage statistics, Resource intensive SQL statements. This report is quite large and comprehensive.


The following instructions are best followed when using Oracle SQL Developer.


Generating an AWR Report:
  •       Get the database ID.

  •       Select the Snap ID for the corresponding start and end time.
  •       The instance number will always be 1 on a single instance, while in a RAC environment it will be 1, 2, 3... depending on the number of nodes you have. Also in a RAC environment, you should generate one report per node. So all you do is change the Instance number and rerun the SQL.
  •      You DON'T need to change connection to the node, just use the SAME connection.
  •      Enter the details obtained above into the parameters for DBMS_WORKLOAD_REPOSITORY.AWR_REPORT_HTML
  •      Highlight the line press [F9] to run.
  •      In the Script Output tab, press [Ctrl]+[A] to select all the output and then [Ctrl]+[Ins] to copy.
  •      Then paste into a text file and rename the file extension to HTM (or HTML).

-- Return a list of all AWR snaps shots.
-- You need to select the snap_id for the start and end period.
SELECT snap_id start1, begin_interval_time, end_interval_time FROM dba_hist_snapshot ORDER BY 1 asc;

-- SELECT OUTPUT FROM TABLE(DBMS_WORKLOAD_REPOSITORY.AWR_REPORT_HTML(<dbid>, <instance>, <start_id>, <end_id>));
-- Insert the database ID, the instance number and the start and end snap_ids.
-- The instance number will always be 1 on a single instance, while in a RAC environment it will be 1, 2, 3... depending on the number of nodes.
-- Highlight line below and run with [F9] then use Ctrl+A on the output, press Ctrl+Ins to copy and paste into a HTML file.
SELECT OUTPUT FROM TABLE(DBMS_WORKLOAD_REPOSITORY.AWR_REPORT_HTML(2321251481, 1, 60943, 60945));



Generating an ASH Report:

  •     Get the database ID.
  •     The instance number will always be 1 on a single instance, while in a RAC environment it will be 1, 2, 3... depending on the number of nodes you have. Also in a RAC environment, you should generate one report per node. So all you do is change the Instance number and rerun the SQL.
  •      You DON'T need to change connection to the node, just use the SAME connection.
  •      Enter the details obtained above into the parameters for DBMS_WORKLOAD_REPOSITORY.ASH_REPORT_HTML
  •      Highlight the line press [F9] to run.
  •      In the Script Output tab, press [Ctrl]+[A] to select all the output and then [Ctrl]+[Ins] to copy.
  •      Then paste into a text file and rename the file extension to HTM (or HTML).






-- Insert the database ID, the instance number and the start and end times.
-- Highlight line below and run with [F9] then use Ctrl+A on the output, press Ctrl+Ins to copy and paste into a HTML file.
SELECT OUTPUT FROM TABLE(DBMS_WORKLOAD_REPOSITORY.ASH_REPORT_HTML(2321251481, 1,
    TO_DATE('30/07/2014 13:52','DD/MM/YYYY HH24:MI'),
    TO_DATE('30/07/2014 14:00','DD/MM/YYYY HH24:MI')));


Sunday, 31 July 2016

FRACTURED BLOCK IN ORACLE

A block in which the header and footer are not consistent at a given SCN. In a user-managed backup, an operating system utility can back up a datafile at the same time that DBWR is updating the file. It is possible for the operating system utility to read a block in a half-updated state, so that the block that is copied to the backup media is updated in its first half, while the second half contains older data. In this case, the block isfractured.

For non-RMAN backups, the ALTER TABLESPACE ... BEGIN BACKUP or ALTER DATABASE BEGIN BACKUP command is the solution for the fractured block problem. When a tablespace is in backup mode, and a change is made to a data block, the database logs a copy of the entire block image before the change so that the database can reconstruct this block if media recovery finds that this block was fractured.

The block that the operating system reads can be split, that is, the top of the block is written at one point in time while the bottom of the block is written at another point in time. If you restore a file containing a fractured block and Oracle reads the block, then the block is considered a corrupt.

What Happens When A Tablespace/Database Is Kept In Begin Backup Mode (Doc ID 469950.1)


Monday, 27 June 2016

DBMS_REDEFINITION

Convert non-partition table to partition table using Oracle redefinition.

Goals
1.       To convert the non-partition table to partition table
2.      No/minimal application downtime
3.      Dependent objects should also get created
4.      Option to convert current non-primary key global index to local indexes
5.       
I’ve used scott schema to redefine a table. Scott will need special privileges for using DBMS_REDEFINITION Package
SQL> grant create any table, alter any table, drop any table,lock any table, select any table, create any trigger, create any index to scott;
Check the existing table & it’s associated attributes
desc EMP
Name                                      Null?    Type
----------------------------------------- -------- -------------
EMPNO                                     NOT NULL NUMBER
NAME                                               VARCHAR2(100)
DEPTNO                                    NOT NULL NUMBER
SQL> select table_name, index_name from user_indexes
/
TABLE_NAME                     INDEX_NAME
------------------------------ ------------------------------
EMP                            IDX_DEPTNO
EMP                            SYS_C004149
SQL> select  constraint_name, constraint_type, table_name 
from user_constraints
/
CONSTRAINT_NAME                C TABLE_NAME
------------------------------ - ------------------------------SYS_C004149                    P EMP
SYS_C004150                    C EMP
Now perform the following steps to redefine the table
STEP 1
Check to make sure that the table can use the online redefinition feature
SQL> EXEC DBMS_REDEFINITION.CAN_REDEF_TABLE('SCOTT','EMP');
PL/SQL procedure successfully completed.

STEP 2
Create a interim partitioned table  (this table will eventually become actual table)

SQL> create table tmp_emp (empno number, ename varchar2(100), 
deptno number) 
partition by list (deptno) 
partition p_10 values (10), 
partition p_20 values (20), 
partition p_30 values (30),
partition p_40 values (40),
partition p_50 values (50),
partition p_60 values (60))
/
I’m creating a local index on the interim table & not copying as part of dependent objects, which is a global index.
SQL> create index idx_deptno_lcl on tmp_emp(deptno) local
/
SQL> desc tmp_emp
Name                                      Null?    Type
----------------------------------------- -------- --------------
EMPNO                                              NUMBER
NAME                                               VARCHAR2(100)
DEPTNO                                             NUMBER
SQL> select table_name, partition_name high_value from user_tab_partitions
/
TABLE_NAME                     HIGH_VALUE
------------------------------ ------------------------------
TMP_EMP                        P_60
TMP_EMP                        P_50
TMP_EMP                        P_40
TMP_EMP                        P_30
TMP_EMP                        P_20
TMP_EMP                        P_10

6 rows selected.

STEP 3
Start the online redefinition process
SQL>EXEC DBMS_REDEFINITION.START_REDEF_TABLE ('SCOTT','EMP','TMP_EMP');
PL/SQL procedure successfully completed.

STEP 4
Copy dependent objects
We will use DBMS_REDEFINITION.COPY_TABLE_DEPENDENTS
SQL> VARIABLE v_num NUMBER;
SQL> EXEC DBMS_REDEFINITION.COPY_TABLE_DEPENDENTS ('SCOTT','EMP','TMP_EMP', 0,TRUE,TRUE,TRUE,FALSE,:v_num,FALSE);

PL/SQL procedure successfully completed.
SQL> PRINT  v_num
 V_NUM
----------         
0

Check the associated attributes for interim table
SQL>  select table_name, index_name from user_indexes
/
TABLE_NAME                     INDEX_NAME
------------------------------ ------------------------------
TMP_EMP                        IDX_DEPTNO_LCL
TMP_EMP                        TMP$$_SYS_C0041490
EMP                            IDX_DEPTNO
EMP                            SYS_C004149
SQL> select  constraint_name, constraint_type, table_name from user_constraints
/
CONSTRAINT_NAME                C TABLE_NAME
------------------------------ - ------------------------------
TMP$$_SYS_C0041370             P TMP_EMP
SYS_C004137                    P EMP
SYS_C004138                    C EMPTMP
$$_SYS_C0041380             C TMP_EMP

STEP 5
Resync the table. This will copy the initial data from the EMP to TMP_EMP table.

SQL> EXEC DBMS_REDEFINITION.SYNC_INTERIM_TABLE('SCOTT','EMP','TMP_EMP');

PL/SQL procedure successfully completed.
SQL> select count(*) from emp;
COUNT(*)
----------        
91
SQL> select count(*) from tmp_emp
/
 COUNT(*)
----------        
91

Now add some rows in the original table EMP.
SQL> set serveroutput on 
declarev1 varchar2(100);
begin
for i in 101..10000
loop
v1:='insert into emp 
values ('||i||','||chr(39)||'ename'||i||chr(39)||',60)';
execute immediate v1;
end loop;
exception
when others then
dbms_output.put_line(substr(sqlerrm,1,50));
end;
/
SQL> select count(*) from emp
/
 COUNT(*)
----------      
9991
SQL>  select count(*) from tmp_emp
/
COUNT(*)
----------        
91
Again resync the table to make sure that the data is correctly getting copied to interim table
SQL> EXEC DBMS_REDEFINITION.SYNC_INTERIM_TABLE('SCOTT','EMP','TMP_EMP');

PL/SQL procedure successfully completed.
SQL>  select count(*) from tmp_emp
/
COUNT(*)
----------      
9991

STEP 6
Finish the online redefinition. Here the redefinition process will be over and interim table will become original table.
SQL> select count(*) from tmp_emp
/
 COUNT(*)
----------
      9991
Just to simulate the load on original table, I’ve added another set of rows from one session & started the finish procedure from another session. Till the point, the insert is not complete & commit is not issued, finish procedure will wait.  So ideally one should execute this procedure, when load on the system is low.
declare
v1 varchar2(100);
begin
for i in 10001..20000
loop
v1:='insert into emp 
values ('||i||','||chr(39)||'name'||i||chr(39)||',60)';
execute immediate v1;
end loop;
exception
when others then
dbms_output.put_line(substr(sqlerrm,1,50));
end;
/
SQL> commit;
Commit complete.
SQL> EXEC DBMS_REDEFINITION.FINISH_REDEF_TABLE('SCOTT','EMP','TMP_EMP');
SQL>  select count(*) from tmp_emp;
 COUNT(*)
----------    
19991
SQL> desc emp 
Name                                      Null?    Type 
----------------------------------------- -------- ------------- 
EMPNO                                              NUMBER 
NAME                                               VARCHAR2(100) 
DEPTNO                                             NUMBER
SQL> select table_name, index_name from user_indexes
/
TABLE_NAME                     INDEX_NAME
------------------------------ ------------------------------
TMP_EMP                        IDX_DEPTNO
TMP_EMP                        SYS_C004149
EMP                            IDX_DEPTNO_LCL
EMP                            TMP$$_SYS_C0041490
SQL> select  constraint_name, constraint_type, table_name from user_constraints;
CONSTRAINT_NAME                C TABLE_NAME
------------------------------ - ------------------------------
SYS_C004149                    P EMP
TMP$$_SYS_C0041490             P TMP_EMP
SYS_C004150                    C EMPTMP
$$_SYS_C0041500             C TMP_EMP

STEP 7
Cross verify the table constraints

SQL> insert into emp values (11,'dsdsd',60);

insert into emp values (11,'dsdsd',60)*ERROR at line 1:
ORA-00001: unique constraint (SCOTT.SYS_C004149) violated
SQL>  insert into emp values (NULL,'dsdsd',60); 

insert into emp values (NULL,'dsdsd',60)  *ERROR at line 1:
ORA-01400: cannot insert NULL into ("SCOTT"."EMP"."EMPNO")

SQL> insert into emp values (11111,'lllll',NULL);

insert into emp values (11111,'lllll',NULL)  *ERROR at line 1:
ORA-01400: cannot insert NULL into ("SCOTT"."EMP"."DEPTNO")

STEP 8
Rename the constraints to required a meaningful name
SQL> ALTER INDEX TMP$$_SYS_C0041540 rename to pk_empno; 
If we want to abort the redefinition process in between, we can use abort procedure
SQL> EXEC DBMS_REDEFINITION.ABORT_REDEF_TABLE ('SCOTT','EMP','TMP_EMP',NULL);

Limitations to the redefinition

1.       Cannot belong to SYS or SYSTEM Schema.
2.      The table to be redefined online should not be using User-defined data types
3.      Should not be a clustered table.
4.      Should not have MV Logs or MVs defined
5.      Table owner cannot be changed, as part of re-definition
6.      Additional space requirement for the interim table (same as original table)



Error / Solutions

SQL> insert into scott.emp values(123,'Demo','Clerk',321,sysdate,2000,10,10);
insert into scott.emp values(123,'Demo','Clerk',321,sysdate,2000,10,10)
                  *
ERROR at line 1:
ORA-04098: trigger 'SCOTT.TEST_EMP' is invalid and failed re-validation


Solution:-

SQL>  select TRIGGER_NAME,TABLE_OWNER,TABLE_NAME from user_triggers

SQL> drop trigger TEST_EMP;

Trigger dropped.

SQL> select * from user_triggers;