Showing posts with label query. Show all posts
Showing posts with label query. Show all posts

Thursday, October 27, 2011

Calculations with key date

A bit of follow up to the previous post ... we'd like to have the key date at hand for performing some calculations, unfortunately we didn't find an easy way so with some micro abap we managed to get that variable into another variable (??) that can be used.
Coding for variables (CMOD on BW side, EXIT_SAPLRRS0_001, include ZXRSRU01 etc.) follows:
Wish there was a simpler way ...
Code:
case i_vnam.
 when 'YV_DAY_2_VAR'.
    if i_step = 2.
      loop at i_t_var_range into loc_var_range where vnam = 'YV_DAY'.
 clear: l_s_range. 
       l_s_range-low = loc_var_range-low.
       l_s_range-sign = 'I'.
       l_s_range-opt = 'EQ'.
 append l_s_range to e_t_range.
      endloop.
    endif.

Monday, September 08, 2008

Random selection, with a bias ...

Say you want to randomly select your employee of the month, but not so randomly, better, you'd like to give your best employees a bigger chance to be selected based on their rating.
This is just an example, you could be randomly displaying ads from your customers, but giving an higher chance to be displayed to those who are paying more, there can be a million other example, but I hope you got the sense of this.
In other terms this means to add some skew to your data, so that it's distribution is no more uniform.
Apart from the statistical implications of this, let's see the sql in action, I'll be using Firebird's Employee sample database, first of all we'll add a column to the employee table, holding the employee rating, then set the rating to 5 for employee 2, 10 for employee 4, 15 for employee 5, 1 for all other employees.
Now that we have rated each employee we need the magic to have them randomly selected with a bias ...
This is accomplished with a "sequence table" which will do just one thing, hold a list of numbers starting from the minimum rating value to the max rating (you'll probably want to preload it with a lot of numbers starting from 1).
Here is the sql:

CREATE TABLE CONSECUTIVE_NUMBER(
NUM Integer NOT NULL,
CONSTRAINT PK_CONSECUTIVE_NUMBER_1 PRIMARY KEY (NUM)
);

And load it:

insert Into Consecutive_number (num) Values (1);
Insert Into Consecutive_number (num) Values (2);
Insert Into Consecutive_number (num) Values (3);
Insert Into Consecutive_number (num) Values (4);
Insert Into Consecutive_number (num) Values (5);
Insert Into Consecutive_number (num) Values (6);
Insert Into Consecutive_number (num) Values (7);
Insert Into Consecutive_number (num) Values (8);
Insert Into Consecutive_number (num) Values (9);
Insert Into Consecutive_number (num) Values (10);
Insert Into Consecutive_number (num) Values (11);
Insert Into Consecutive_number (num) Values (12);
Insert Into Consecutive_number (num) Values (13);
Insert Into Consecutive_number (num) Values (14);
Insert Into Consecutive_number (num) Values (15);
Insert Into Consecutive_number (num) Values (16);
Insert Into Consecutive_number (num) Values (17);
Insert Into Consecutive_number (num) Values (18);
Insert Into Consecutive_number (num) Values (19);
Insert Into Consecutive_number (num) Values (20);

Now we are ready for the magic, we'll be joining our employee table with this sequence table, generating a specific cross join which will hold as many rows for each employee as it's rating, thus, when doing a random select on this data there will be a bigger chance to choose some employees, i.e. employee number 5 will have 15 times the chance to be selected than employee number 8 and 3 times the chances of employee number 2.
Here is the query, embedded in a view:

CREATE VIEW
RANDOM_SKEWED_EMPLOYEE (EMP_NO)
AS
/* write select statement here */
SELECT
FIRST 1
a.EMP_NO
FROM
EMPLOYEE a
INNER JOIN
CONSECUTIVE_NUMBER c
ON
c.NUM BETWEEN 1 AND a.RATING
ORDER BY
rand();

The join clause does the magic, as I said.
Now extract a small sample:

24
5
36
85
5
2
12
24
144
5
5
20
94
5
4
127
5
94
85
4
2
34
136
4
9

Let's see it's distribution, even on such a small sample you'll see that those 3 employees appear more often than others and with more or less the expected relative ratio:
Emp_no Times
2 5
4 9
5 15
9 4
14 1
20 1
28 2
29 2
36 1
37 1
52 2
71 1
72 1
83 1
113 1
114 1
118 2
121 1
127 1
138 2
144 1

Of course a larger sample will be closer to the desired result.
This example is buit on Firebird, but it should work in any database.
HTH

Note that PostgreSQL's generate_series() instruction might help a lot ;)

Tuesday, July 22, 2008

Has them all

A question that pops up frequently on Devshed forums is "How can I get all products that are available in Red and Green colors?" or "How can I find out which customers bought this book and that CD?", solution is simple and I'll provide an example here, it can be made more complicate at your option, but it all boils down to a where and an having condition.
Say we have a table that lists all products and the colors in which those products are available:

  1. CREATE TABLE PRODUCT_COLORS(
  2. PRODUCT_CODE CHAR(5) NOT NULL,
  3. COLOR_CODE CHAR(1) NOT NULL,
  4. CONSTRAINT PRODUCT_COLORS_PK PRIMARY KEY (PRODUCT_CODE,COLOR_CODE)
  5. );

Data looks like:

Code:
XXXXX                R
XXXXX B
YYYYY Y
YYYYY G
ZZZZZ G
ZZZZZ R
First column is the product code, second column is the color code.
Say we want to know which products are available in G(reen) and R(ed), the query is simple, we'll list all products which do have the Red or Green option and then filter out all those that don't have both, getting the desired result (in this case product 'ZZZZZ')
See it in action:

  1. SELECT a.PRODUCT_CODE
  2. FROM PRODUCT_COLORS a
  3. WHERE a.COLOR_CODE IN ('R', 'G')
  4. GROUP BY a.PRODUCT_CODE
  5. HAVING COUNT(a.PRODUCT_CODE) = 2

See where I implemented the two conditions? One in the where clause and the second pass to filter out all products which don't have both in the having clause.
Example is built on Firebird, but should work in MySQL, PostgreSQL or any other mainstream database too.

Monday, July 21, 2008

Another great new feature of Firebird 2.5

Other than CREATE USER something really valuable has been added, the ability to query other databases.
Using a table structure like my previous post what follows is an example of a query running on an external database:

SET TERM ^ ;
CREATE PROCEDURE GET_MASTER_PROD_ALL_EXT
RETURNS (
P_CODE Char(5),
I_ENABLED Char(1),
P_DESCR Varchar(50) )
AS
declare variable qry varchar(5000);
BEGIN
qry = 'SELECT pm.PRODUCT_CODE, pm.IS_ENABLED, pm.PRODUCT_DESCR
FROM product_master pm';
EXECUTE STATEMENT qry ON EXTERNAL DATA SOURCE 'localhost:c:\pippo2.fdb'
AS USER 'sysdba' PASSWORD 'masterkey'
INTO :p_code, :i_enabled, :p_descr;
SUSPEND;
END^
SET TERM ; ^

The trick is the ON EXTERNAL DATA SOURCE clause, which allows you to specify a target database ("localhost:c:\pippo2.fdb" in my case, generally that's time to start using aliases) and the user/password to be used
As you can see , no "dblinks" yet, but of course you can embed it into a view and join it (the stored proc or the view) with other local or remote objects.
The querystring can also be built dynamically passing parameters.
Isn't it great?

Sunday, April 06, 2008

Two basic indexing tips ...

Here are two basic tips for proper indexing ...

  1. Don't mess with datatypes, too often people refer to an attribute defining it as one datatype in a table and as another in different tables, this actually prevents index usage in joins (forget about FKs for this time ;)) See an example here. You could declare a function based index as a workaround, but why don't we all try to make it right?
  2. Put indexes where the database can really use them, if a table is to be fully scanned anyway, it's indexes are unlikely to be used, unless you can compare those index entries with other indexes on tables that won't be fully scanned. Ordering is another game ;). See here for an example.
Easy, isn't it? But these mistakes are still very common ...

Thursday, November 02, 2006

Of correlated and uncorrelated subqueries

Many SQL questions on Devshed forums are easily solved using correlated and uncorrelated subqueries, but many don't know of them at all, or at least are not shure about the syntax, usage and performance implications, they sometimes rave about a stored procedure to loop and select other values ...
The definition (taken from O'Reilly's "Learning SQL on SQL Server 2005") of a correlated subquery:
A correlated subquery is an inner subquery whose information is referenced by the main, outer query such that the inner query may be thought of as being executed repeatedly.
And the definition of an uncorrelated subquery:
A noncorrelated subquery is a subquery that is independent of the outer query. In other words, the subquery could be executed on its own.
This is all fine and dandy, but what's the use?
I'll put up two simple examples of usage, which should make clear which kind of questions can be answered with (un)correlated subqueries:

Example 1: get the number of "open" orders, the number of "waiting" orders and the open vs. waiting ratio

  1. SELECT
  2. COUNT(*) AS o_open,
  3. (SELECT COUNT(*) FROM sales WHERE order_status = 'waiting') AS o_waiting,
  4. (COUNT(*))/
  5. (SELECT COUNT(*) FROM sales WHERE order_status = 'waiting') AS open_waiting_ratio
  6. FROM SALES WHERE order_status = 'open';

You can see that the queries are uncorrelated, and they better be, as the where conditions are different.

Example 2: we want to see the customer number, the amount of the customer's last order and the total amount ordered by each customer

  1. SELECT
  2. s.cust_no,
  3. s.total_value AS order_value,
  4. (SELECT SUM(i_s.total_value) FROM sales i_s WHERE i_s.cust_no = s.cust_no) total_ordered
  5. FROM sales s
  6. WHERE
  7. s.order_date =
  8. (SELECT MAX(ii_s.order_date) FROM sales ii_s WHERE ii_s.cust_no = s.cust_no);

You can see that both subqueries are correlated, the first one gets total amount ordered for each customer, the second one gets the last order date (field is datetime) and helps determine the last order in the main query.

This should make you aware that some kind of set based logic is often disguised as procedural and usually leads to ugly cursor usage in stored procedures instead of direct queries by the inexperienced.

Another example of a correlated subquery to display customers and one randomly selected purchase order for each of them:

  1. SELECT
  2. a.CUST_NO,
  3. a.CUSTOMER,
  4. b.PO_NUMBER
  5. FROM
  6. CUSTOMER a
  7. INNER JOIN
  8. sales b
  9. ON a.CUST_NO = b.CUST_NO
  10. WHERE
  11. b.PO_NUMBER =
  12. (
  13. SELECT
  14. first 1 c.PO_NUMBER
  15. FROM
  16. sales c
  17. WHERE
  18. c.CUST_NO = b.CUST_NO
  19. ORDER BY
  20. rand()
  21. )


Both examples are run against Firebird's standard EMPLOYEE.FDB.


Correlated subqueries are also useful to specify updates and deletes on a table conditioned to values of another table (instead of joining them) see this example:

  1. UPDATE
  2. stops s
  3. SET
  4. s.inservice=1
  5. WHERE EXISTS
  6. (
  7. SELECT
  8. ls.stopid
  9. FROM
  10. linestop ls
  11. WHERE
  12. s.stopid=ls.stopid
  13. AND
  14. ls.signid=79
  15. );

Saturday, October 28, 2006

Firebird's explain PLAN

After a post on Devshed forums I'm putting here an excerpt from a little tutorial about Firebird I wrote some time ago, this is about the explain plan of a query:

More on speeding … the Explain plan

An extremely useful feature of relational databases is the “explain plan”,which
shows us how the database engine is going to execute a query. It allows the DBA to
check if indexes are used and how tables are queried. One of the most effective ways of
using it is to use the Firebird option “PLANONLY” which means that the query to be
analyzed is submitted to the server and the plan retrieved, but the query is not executed!

Note that this is also an effective way of checking the SQL syntax of a statement, much
like Oracle’s parse statement.

The planonly option can be set to ON or OFF at your option, remember to turn it
off to obtain results for your queries.

The plan is accessed through ISQL this way:

>isql
SQL> CONNECT “C:\firedata\first_db.fdb” user ‘SYSDBA’ password ‘guess_me’;

SQL> SET PLANONLY ON;
SQL> SELECT a.emp_no, b.currency from employee a inner join country b on
a.job_country = b.country;

After submitting the query we retrieve the plan, which shows usage of the primary key of
table country:

PLAN JOIN (A NATURAL,B INDEX (RDB$PRIMARY1))
SQL> SET PLANONLY OFF;

This way we can check the execution of more complex queries and find areas of
improvement

You can see an example of it's usage here.