SQL Server, statistics, performance and the Ascending key problem

Hi guys,

September is coming and it is time for our editorial staff to reopen its doors. I hope you have recharged your batteries to 100%.

Today we will talk about statistics and performance. Yes, we have talked about it many times before but today we will focus on an issue that could lead to a poor performance situation. 

Maybe your query is poor performing due to this issue?

Ready? Go!

Statististics and the Ascending Key Problem

Statistics need to be updated because they help SQL server create a good execution plan. I like to do an example by making a parallel between extracting data from the database and a trip: if i have to go to work and my office is near me it is better to take a bicycle. If my work is far from me it is better take the car or the train.

Likewise, if I need to extract a few records, the optimizer can implement a JOIN by choosing a nested loop. Otherwise it would be better to use other operators such as merge or hash.

This is why statistics are important. Because updating statistics is a time-consuming process, statistics are not updated with every change made to a table.

The problem arises when you have a table with a clustered index.

So, suppose we have a table with a few million rows.

Now, for example, let's develop a procedure that inserts a few hundred rows.

Statistics are not updated because the number of rows inserted is lower than the threshold used by SQL Server.

What happens?

It happens that the estimated cardinality is 1 and not the correct one of a few hundred.

This problem could lead to the creation of a not good execution plan.

There are some traceflags to change the threshold used by SQL Server and starting with SQL Server 2016 onwards things are changed again.

But this post was written to clarify a logic that could lead to a poor performance situation.

As usual, a good example is the best way to understand how things work!

Example

Let's take as an example a table of the SAP Business One Erp2 (This ERP supports two different database engines: SQL Server and the proprietary in memory rdbms SAP HANA )

We choose for example the table OACT. 

SAP BUSINESS ONE OACT TABLE

The table OACT has a clustered index on the AcctCode field:

SAP BUSINESS ONE OACT_PRIMARY CLUSTERED INDEX

Executing the command


DBCC SHOW_STATISTICS('OACT', OACT_PRIMARY)

We can see that statistics are updated to 8th of January 2021

 

This table has 568 rows.

Let's now insert 50 Rows (50/568 equal to 8%)

       
INSERT INTO [dbo].[oact]
            ([acctcode],
             [acctname],
             [currtotal],
             [endtotal],
             [finanse],
             [groups],
             [budget],
             [frozen],
             [free_2],
             [postable],
             [fixed],
             [levels],
             [exportcode],
             [grpline],
             [fathernum],
             [accntntcod],
             [cashbox],
             [groupmask],
             [ratetrans],
             [taxincome],
             [exmincome],
             [extrmatch],
             [intrmatch],
             [acttype],
             [transfered],
             [blnctrnsfr],
             [overtype],
             [overcode],
             [sysmatch],
             [prevyear],
             [actcurr],
             [ratedifact],
             [systotal],
             [fctotal],
             [protected],
             [realacct],
             [advance],
             [createdate],
             [updatedate],
             [frgnname],
             [details],
             [extrasum],
             [project],
             [revalmatch],
             [datasource],
             [locmth],
             [mthcounter],
             [bnkcounter],
             [usersign],
             [locmantran],
             [loginstanc],
             [objtype],
             [validfor],
             [validfrom],
             [validto],
             [validcomm],
             [frozenfor],
             [frozenfrom],
             [frozento],
             [frozencomm],
             [counter],
             [segment_0],
             [segment_1],
             [segment_2],
             [segment_3],
             [segment_4],
             [segment_5],
             [segment_6],
             [segment_7],
             [segment_8],
             [segment_9],
             [formatcode],
             [cfwrlvnt],
             [exchrate],
             [revalacct],
             [lastrevbal],
             [lastrevdat],
             [dfltvat],
             [vatchange],
             [category],
             [transcode],
             [overcode5],
             [overcode2],
             [overcode3],
             [overcode4],
             [dflttax],
             [taxpostacc],
             [acctstrle],
             [meaunit],
             [baldirect],
             [usersign2],
             [plnglevel],
             [multilink],
             [prjrelvnt],
             [dim1relvnt],
             [dim2relvnt],
             [dim3relvnt],
             [dim4relvnt],
             [dim5relvnt],
             [accrualtyp],
             [datevacct],
             [datevautoa],
             [datevfirst],
             [snapshotid],
             [pcn874rpt],
             [scadjust],
             [bplid],
             [bplname],
             [subledgern],
             [vatregnum],
             [actid],
             [closingacc],
             [purpcode],
             [refcode],
             [blocmanpos],
             [priacccode],
             [cstacconly],
             [allowefrom],
             [allowedto],
             [balancea],
             [rmrktmpt],
             [cemrelvnt],
             [cemcode],
             [stdactcode],
             [taxoncode],
             [inclasstyp],
             [inclasscat],
             [exclasstyp],
             [exclasscat])
SELECT TOP 50 'Z' + LEFT(Ltrim([acctcode]), 14),
              [acctname],
              [currtotal],
              [endtotal],
              [finanse],
              [groups],
              [budget],
              [frozen],
              [free_2],
              [postable],
              [fixed],
              [levels],
              [exportcode],
              [grpline],
              [fathernum],
              [accntntcod],
              [cashbox],
              [groupmask],
              [ratetrans],
              [taxincome],
              [exmincome],
              [extrmatch],
              [intrmatch],
              [acttype],
              [transfered],
              [blnctrnsfr],
              [overtype],
              [overcode],
              [sysmatch],
              [prevyear],
              [actcurr],
              [ratedifact],
              [systotal],
              [fctotal],
              [protected],
              [realacct],
              [advance],
              [createdate],
              [updatedate],
              [frgnname],
              [details],
              [extrasum],
              [project],
              [revalmatch],
              [datasource],
              [locmth],
              [mthcounter],
              [bnkcounter],
              [usersign],
              [locmantran],
              [loginstanc],
              [objtype],
              [validfor],
              [validfrom],
              [validto],
              [validcomm],
              [frozenfor],
              [frozenfrom],
              [frozento],
              [frozencomm],
              [counter],
              [segment_0],
              [segment_1],
              [segment_2],
              [segment_3],
              [segment_4],
              [segment_5],
              [segment_6],
              [segment_7],
              [segment_8],
              [segment_9],
              [formatcode],
              [cfwrlvnt],
              [exchrate],
              [revalacct],
              [lastrevbal],
              [lastrevdat],
              [dfltvat],
              [vatchange],
              [category],
              [transcode],
              [overcode5],
              [overcode2],
              [overcode3],
              [overcode4],
              [dflttax],
              [taxpostacc],
              [acctstrle],
              [meaunit],
              [baldirect],
              [usersign2],
              [plnglevel],
              [multilink],
              [prjrelvnt],
              [dim1relvnt],
              [dim2relvnt],
              [dim3relvnt],
              [dim4relvnt],
              [dim5relvnt],
              [accrualtyp],
              [datevacct],
              [datevautoa],
              [datevfirst],
              [snapshotid],
              [pcn874rpt],
              [scadjust],
              [bplid],
              [bplname],
              [subledgern],
              [vatregnum],
              'Z' + LEFT(Ltrim([acctcode]), 14) AS [ActId],
              [closingacc],
              [purpcode],
              [refcode],
              [blocmanpos],
              [priacccode],
              [cstacconly],
              [allowefrom],
              [allowedto],
              [balancea],
              [rmrktmpt],
              [cemrelvnt],
              [cemcode],
              [stdactcode],
              [taxoncode],
              [inclasstyp],
              [inclasscat],
              [exclasstyp],
              [exclasscat]
FROM   oact
WHERE  acctcode <> '100000000000000'

To verify that statistics are not changed type again the command:


DBCC SHOW_STATISTICS('OACT', OACT_PRIMARY)

Now type the command


SELECT * FROM OACT where AcctCode >= 'A00000000000000'

While 51 rows are returned watching to the property we can clearly see that the estimated number of lines to read is 1.

 

This is the problem that could lead to the creation of an not optimal execution plan!


Using the Traceflag 2371

So, how to mitigate this problem?
Using the traceflag 2371, SQL Server uses a decreasing and dynamic statistics update threshold.
This will result in a lower statistical update threshold as cardinality increases.

Starting from SQL Server 2016 this behaviour is enabled by default.

This is a summary:

 
Table type Table cardinality Up to SQL2014 (12.x) Recompilation threshold (# modifications) or TF 2371
 From SQL2016 (13.x) Recompilation threshold (# modifications)
Temporary n<6 td=""> 6 6
Temporary 6<=n<=500 500 500
Permanent n<=500 500 500
Temporary or Permanent n>=500 500 + (0.20 * n) Min between 500 + (0.20 * n) and SQRT(1000 + n)


Enable this trace flag also if you are using SQL Server on  a SAP ERP system.



I hope you found the reading interesting and if so follow me!

Luca Biondi @ SQLServerPerformance blog 2019-2021





 

 

 

 
Next post:

Comments

Post a Comment

I Post più popolari

SQL Server, execution plan and the lazy spool (clearly explained)

Image
Hi Guys, Welcome back! In one of the last posts we talked about Table Spool . We had explained what Spool are for by telling the somewhat curious story of the halloween problem . Here we specifically talked about a type of spool called Eager Spool   You can look to the execution plan below where the eager Spool is used to avoid the halloween problem in classic query of the "salary increase".. i suggest Well, if the story of the eager spool intrigued you, today instead we will speak about another type of Spool, the Lazy Spool . The Lazy spool Let's take a small step back. So, what is a Spool operation?  As we have already understood a spool operation is simply a temporary storage of data . Data are read from a source table and stored inside a worktable in the Tempdb database . While an Eager Spool is used to prevent the Halloween problem, the Lazy Spool is instead used to store data that will later be needed again when running a query . So, when a Lazy spool is used?  ...
Read more

SQL Server, datetime vs. datetime2

Image
Hi guys, This time, in this post we will speak about datetime and datetime2 datatypes. We will talk about the pros and cons. We will talk about all the differences between the two types of data. Write me in the comments which type of data do you prefer between the two. Enjoy the reading!     Datetime Vs. Datetime2 First of all, it must be said that the datatype datetime is certainly the best known native format for storing a date in SQL Server. However, Microsoft has also been introducing a new type of data for many years now, also used to manage dates. It is in fact from SQL server 2008 that it is present in datetime2 data type. This "new" type of data was born primarily to have higher accuracy . Let's in fact talk about ... precision ! Datetime Certainly the datetime data type is not very precise and certainly there are many cases in which its precision is simply not sufficient. Let's see it with an example. Open the SQL Management studio (SSMS) and write the fo...
Read more

La clausola NOLOCK. Approfondiamo e facciamo chiarezza!

Image
Ehi ragazzi! La domanda è di quelle importanti! Siete pronti per scoprire tutto ma proprio tutto sulla clausola NOLOCK?   Se la riposta è si, allora seguitemi! Oggi ne spiegheremo il funzionamento in maniera approfondita. Faremo una introduzione al concetto di tipi di LOCK Ed infine vedremo come mai una SELECT ne può bloccare un'altra. Non vi nascondo che il motivo di questo articolo è fare chiarezza. Su questo argomento si sentono spesso affermazioni “curiose” ma tra le tante questa è la prima: “ uso la SELECT con l’opzione WITH(LOCK) così non blocco le altre tabelle ”.   Ma, è proprio vero quanto affermato? Beh oggi facciamo il punto!   Buona lettura!     La clausola NOLOCK Diciamo subito per iniziare che cos’è la NOLOCK. NOLOCK è il nome di una clausola che è possibile specificare all’interno di uno statement di tipo SELECT utilizzando la seguente sintassi: SELECT * FROM TABELLA WITH (NOLOCK) Si noti che non possiamo ...
Read more