Slow query performance using UPPER, LOWER or RTRIM in SQL Server 2017 and 2019

Hi Guys,
Are you ready for another interesting post?

Today we will inspect and solve a problem that afflict SQL SERVER 2017 and SQL Server 2019 that cause Slow query performance.
Is this also an opportunity to talk about the latest SQL server CU: the CU4

 

The Upper , Lower and Rtrim...

So where is the problem?
Imagine to execute a Query that has a filter predicate on a column (e.g. in our example the field "nemail2").
Suppose also using the UPPER, LOWER, or RTRIM functions.
In this scenario you may have an inaccurate cardinality estimates and so SQL Server may uses an inefficient execution plan.

We already talked about cardinality estimates here: Inside the SQL Server Query Optimizer - part 3 Cardinality estimation 

Let's go with our usual example.

Take a database with compatibity level 140 or 150.
Now filter on a column of nvarchar data type for example:


SELECT id FROM Person WHERE nemail LIKE (N'%@libero.it') 



SELECT id FROM Person WHERE nemail LIKE UPPER(N'%@libero.it')

 

As you can see if you use a UPPER (and also a LOWER and RTRIM) function the estimated number of rows if not accurate.

Estimated rows infacts are 576 with 14 rows extracted while if you don't use any function estimated rows are 19.

As said before this could cause slow query performances.
But..

How To Solve this problem?


The only method I found is to force the "legacy cardinality estimation" using an hint as below: 
       
SELECT id FROM Persone WHERE nemail LIKE UPPER(N'%@libero.it')
OPTION(USE HINT('FORCE_LEGACY_CARDINALITY_ESTIMATION'))


Any other method doensn’t work: 

SELECT id FROM Person WHERE nemail LIKE UPPER(N'%@libero.it')
OPTION(USE HINT('ENABLE_QUERY_OPTIMIZER_HOTFIXES')) 

SELECT id FROM Person WHERE nemail LIKE UPPER(N'%@libero.it') 
OPTION(QUERYTRACEON 4199)

This is infact a bug but it is already fixed with the latest CU (cumulative update) for SQL Server 2017 and SQL Server 2019.

So now we will install the latest CU for SQL Server 2019: the CU4

Install the latest SQL Server CU

Install a CU is an easy operation.
Download it through the official Microsoft site.



Execute it with "Run As Administrator"..


The SQL Server engine service will be restarted and the upgrade is done.

Now run the test again

Now, if you execute the Query below: 
       
SELECT id FROM Persone WHERE nemail LIKE UPPER(N'%@libero.it')


The number of estimated rows are once again wrong:



BUT


If you now specificy the following option.. 

       
SELECT id FROM Persone WHERE nemail LIKE UPPER(N'%@libero.it')
OPTION(USE HINT('ENABLE_QUERY_OPTIMIZER_HOTFIXES'))

or 

       

SELECT id FROM Persone WHERE nemail LIKE UPPER(N'%@libero.it')
OPTION(QUERYTRACEON 4199)

You will obtain a more accurate estimated number of rows (19,242) also using the UPPER, LOWER or RTRIM functions.





That's all for today!
I hope you enjoy this post.

See you soon.
Luca


Luca Biondi @ SQLServerPerformance blog 2020!  
 












Previous post: How to upgrade the SQL Server Management studio. SSMS 18.5 news. Moreover.. Notebook and the SQL assessment API

Comments

Post a Comment

I Post più popolari

SQL Server, execution plan and the lazy spool (clearly explained)

Image
Hi Guys, Welcome back! In one of the last posts we talked about Table Spool . We had explained what Spool are for by telling the somewhat curious story of the halloween problem . Here we specifically talked about a type of spool called Eager Spool   You can look to the execution plan below where the eager Spool is used to avoid the halloween problem in classic query of the "salary increase".. i suggest Well, if the story of the eager spool intrigued you, today instead we will speak about another type of Spool, the Lazy Spool . The Lazy spool Let's take a small step back. So, what is a Spool operation?  As we have already understood a spool operation is simply a temporary storage of data . Data are read from a source table and stored inside a worktable in the Tempdb database . While an Eager Spool is used to prevent the Halloween problem, the Lazy Spool is instead used to store data that will later be needed again when running a query . So, when a Lazy spool is used?  ...
Read more

SQL Server, datetime vs. datetime2

Image
Hi guys, This time, in this post we will speak about datetime and datetime2 datatypes. We will talk about the pros and cons. We will talk about all the differences between the two types of data. Write me in the comments which type of data do you prefer between the two. Enjoy the reading!     Datetime Vs. Datetime2 First of all, it must be said that the datatype datetime is certainly the best known native format for storing a date in SQL Server. However, Microsoft has also been introducing a new type of data for many years now, also used to manage dates. It is in fact from SQL server 2008 that it is present in datetime2 data type. This "new" type of data was born primarily to have higher accuracy . Let's in fact talk about ... precision ! Datetime Certainly the datetime data type is not very precise and certainly there are many cases in which its precision is simply not sufficient. Let's see it with an example. Open the SQL Management studio (SSMS) and write the fo...
Read more

La clausola NOLOCK. Approfondiamo e facciamo chiarezza!

Image
Ehi ragazzi! La domanda è di quelle importanti! Siete pronti per scoprire tutto ma proprio tutto sulla clausola NOLOCK?   Se la riposta è si, allora seguitemi! Oggi ne spiegheremo il funzionamento in maniera approfondita. Faremo una introduzione al concetto di tipi di LOCK Ed infine vedremo come mai una SELECT ne può bloccare un'altra. Non vi nascondo che il motivo di questo articolo è fare chiarezza. Su questo argomento si sentono spesso affermazioni “curiose” ma tra le tante questa è la prima: “ uso la SELECT con l’opzione WITH(LOCK) così non blocco le altre tabelle ”.   Ma, è proprio vero quanto affermato? Beh oggi facciamo il punto!   Buona lettura!     La clausola NOLOCK Diciamo subito per iniziare che cos’è la NOLOCK. NOLOCK è il nome di una clausola che è possibile specificare all’interno di uno statement di tipo SELECT utilizzando la seguente sintassi: SELECT * FROM TABELLA WITH (NOLOCK) Si noti che non possiamo ...
Read more