Running total over date range - fill in the missing dates

Question

I have the following table.

DATE  | AMT
10/10 | 300
12/10 | 300
01/11 | 200
03/11 | 100

How do I get the monthly total? A result like -

DATE | TOT
1010 | 300
1110 | 300
1210 | 600 
0111 | 800
0211 | 800
0311 | 900

A sql statement like

SELECT SUM(AMT) FROM TABLE1 WHERE DATE BETWEEN '1010' AND '0111'

would result in the 800 for 0111 but...

NOTE There is not a date restriction. which is my dilemma. How do I populate this column without doing a loop for all dates and have the missing months displayed as well?

Answer 1

To cater for missing months, create a template table to join against.

Think of it as caching. Rather than looping through and filling gaps, just have a calendar cached in your database.

You can even combine multiple calendars (start of month, start of week, bank holidays, working day, etc) all into one table, with a bunch of search flags and indexes.

You end up with something like...

SELECT
  calendar.date,
  SUM(data.amt)
FROM
  calendar
LEFT JOIN
  data
    ON  data.date >= calendar.date
    AND data.date <  calendar.date + INTERVAL 1 MONTH
WHERE
      calendar.date >= '20110101'
  AND calendar.date <  '20120101'
GROUP BY
  calendar.date

EDIT

I just noticed that the OP wants a running total.

This -is- possible in SQL but it is extremely inefficient. The reason being that the result from one month isn't used to calculate the following month. Instead the whole running-total has to be calculated again.

For this reason It is normally strongly recommended that you calculate the monthly total as above, then use your application to itterate through and make the running total values.

If you really must do it in SQL, it would be something like...

SELECT
  calendar.date,
  SUM(data.amt)
FROM
  calendar
LEFT JOIN
  data
    ON  data.date >= @yourFirstDate
    AND data.date <  calendar.date + INTERVAL 1 MONTH
WHERE
      calendar.date >= @yourFirstDate
  AND calendar.date <  @yourLastDate
GROUP BY
  calendar.date

Answer 2

the main problem is the and have the missing months displayed as well? I don't see how to do it with out an aux table containing the combination of month\year to be displayed:

create table table1(
date datetime,
amt int
)
insert into table1 values ('10/10/2010',100)
insert into table1 values ('12/12/2010',200)
insert into table1 values ('01/01/2011',50)
insert into table1 values ('03/03/2011',500)

truncate table #dates
create table #dates(
_month int,
_year int
)
insert into #dates values(10,2010)
insert into #dates values(11,2010) --missing month
insert into #dates values(12,2010)
insert into #dates values(01,2011)
insert into #dates values(02,2011)--missing month
insert into #dates values(03,2011)


select D._month, D._year, sum(amt)
from #dates D left join TABLE1 T on D._month=month(T.date) and D._year=year(T.date)
group by D._month, D._year

Answer 3

You can also generate a range on the fly, pass its value as the interval to DATE_ADD, and basically project a sequence of month values.

As @Dems said, you need to have a correlated subquery calculate the running total, which will be very inefficient, because it will run a nested loop internally.

To see how to generate the sequence, check my post here: How to generate a range of numbers in Mysql

The end query should look something like this: (Incidentally, you should have a date column, not this varchar mess).

/*NOTE: This assumes a derived table (inline view) containing the sequence of date values and their corresponding TOT value*/
SELECT
  DATEVALUES.DateValue,
  (
  SELECT SUM(TABLE1.AMT) FROM TABLE1 WHERE TABLE1.DateValue <= DATEVALUES.DateValue)
  ) AS RunningSubTotal
FROM
  DATEVALUES

Or something like that.

Answer 4

select sum(AMT) from TABLE1 group by Date