Changes between Version 2 and Version 3 of TracReports


Ignore:
Timestamp:
2023-04-20T18:47:43Z (13 months ago)
Author:
trac
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • TracReports

    v2 v3  
    1 = Trac Reports =
     1= Trac Reports
     2
    23[[TracGuideToc]]
    34
    4 The Trac reports module provides a simple, yet powerful reporting facility
    5 to present information about tickets in the Trac database.
    6 
    7 Rather than have its own report definition format, TracReports relies on standard SQL
    8 `SELECT` statements for custom report definition.
    9 
    10   '''Note:''' ''The report module is being phased out in its current form because it seriously limits the ability of the Trac team to make adjustments to the underlying database schema. We believe that the [wiki:TracQuery query module] is a good replacement that provides more flexibility and better usability. While there are certain reports that cannot yet be handled by the query module, we intend to further enhance it so that at some point the reports module can be completely removed. This also means that there will be no major enhancements to the report module anymore.''
    11 
    12   ''You can already completely replace the reports module by the query module simply by disabling the former in [wiki:TracIni trac.ini]:''
    13   {{{
     5The Trac reports module provides a simple, yet powerful reporting facility to present information about tickets in the Trac database.
     6
     7Rather than have its own report definition format, TracReports relies on standard SQL `SELECT` statements for custom report definition.
     8
     9  '''Note:''' The report module is being phased out in its current form because it seriously limits the ability of the Trac team to make adjustments to the underlying database schema. We believe that the [wiki:TracQuery query module] is a good replacement that provides more flexibility and better usability. While there are certain reports that cannot yet be handled by the query module, we intend to further enhance it so that at some point the reports module can be completely removed. This also means that there will be no major enhancements to the report module anymore.
     10
     11  You can already completely replace the reports module by the query module simply by disabling the former in [wiki:TracIni trac.ini]:
     12  {{{#!ini
    1413  [components]
    1514  trac.ticket.report.* = disabled
    1615  }}}
    17   ''This will make the query module the default handler for the “View Tickets” navigation item. We encourage you to try this configuration and report back what kind of features of reports you are missing, if any.''
    18 
    19   '''''You will almost definitely need to restart your httpd at this point.'''''
     16  This will make the query module the default handler for the “View Tickets” navigation item. We encourage you to try this configuration and report back what kind of features of reports you are missing, if any.
    2017
    2118A report consists of these basic parts:
    22  * '''ID''' -- Unique (sequential) identifier
    23  * '''Title'''  -- Descriptive title
    24  * '''Description'''  -- A brief description of the report, in WikiFormatting text.
    25  * '''Report Body''' -- List of results from report query, formatted according to the methods described below.
    26  * '''Footer''' -- Links to alternative download formats for this report.
    27 
    28 == Changing Sort Order ==
    29 Simple reports - ungrouped reports to be specific - can be changed to be sorted by any column simply by clicking the column header.
    30 
    31 If a column header is a hyperlink (red), click the column you would like to sort by. Clicking the same header again reverses the order.
    32 
    33 == Navigating Tickets ==
    34 Clicking on one of the report results will take you to that ticket. You can navigate through the results by clicking the ''Next Ticket'' or ''Previous Ticket'' links just below the main menu bar, or click the ''Back to Report'' link to return to the report page.
    35 
    36 You can safely edit any of the tickets and continue to navigate through the results using the Next/Previous/Back to Report links after saving your results, but when you return to the report, there will be no hint about what has changed, as would happen if you were navigating a list of tickets obtained from a query (see TracQuery#NavigatingTickets). ''(since 0.11)''
    37 
    38 == Alternative Download Formats ==
    39 Aside from the default HTML view, reports can also be exported in a number of alternative formats.
    40 At the bottom of the report page, you will find a list of available data formats. Click the desired link to
    41 download the alternative report format.
    42 
    43 === Comma-delimited - CSV (Comma Separated Values) ===
     19 * '''ID''' — Unique (sequential) identifier
     20 * '''Title''' — Descriptive title
     21 * '''Description''' — A brief description of the report, in WikiFormatting text.
     22 * '''Report Body''' — List of results from report query, formatted according to the methods described below.
     23 * '''Footer''' — Links to alternative download formats for this report.
     24
     25== Changing Sort Order
     26
     27Simple reports - ungrouped reports to be specific - can be sorted by clicking the column header.
     28
     29If a column header is a hyperlink (red), click the column to sort by it. Clicking the same header again reverses the sort order.
     30
     31== Navigating Tickets
     32
     33Clicking on one of the report results will take you to that ticket. You can navigate through the results by clicking the ''Next Ticket'' or ''Previous Ticket'' contextual navigation links, or click the ''Back to Report'' link to return to the report page.
     34
     35You can safely edit any of the tickets and continue to navigate through the results using the ''!Next/Previous/Back to Report'' links after saving your results, but when you return to the report, there will be no hint about what has changed, in contrast to the query results (see TracQuery#NavigatingTickets).
     36
     37== Alternate Download Formats
     38
     39In addition to the HTML view, reports can also be exported in a number of alternate formats.
     40At the bottom of the report page, you will find a list of available data formats. Click the desired link to
     41download the alternate format.
     42
     43=== Comma-delimited - CSV (Comma Separated Values)
     44
    4445Export the report as plain text, each row on its own line, columns separated by a single comma (',').
    45 '''Note:''' Carriage returns, line feeds, and commas are stripped from column data to preserve the CSV structure.
    46 
    47 === Tab-delimited ===
     46'''Note:''' The output is fully escaped so carriage returns, line feeds, and commas will be preserved in the output.
     47
     48=== Tab-delimited
     49
    4850Like above, but uses tabs (\t) instead of comma.
    4951
    50 === RSS - XML Content Syndication ===
     52=== RSS - XML Content Syndication
     53
    5154All reports support syndication using XML/RSS 2.0. To subscribe to an RSS feed, click the orange 'XML' icon at the bottom of the page. See TracRss for general information on RSS support in Trac.
    5255
    53 ----
    54 
    55 == Creating Custom Reports ==
    56 
    57 ''Creating a custom report requires a comfortable knowledge of SQL.''
    58 
    59 A report is basically a single named SQL query, executed and presented by
    60 Trac.  Reports can be viewed and created from a custom SQL expression directly
    61 in from the web interface.
    62 
    63 Typically, a report consists of a SELECT-expression from the 'ticket' table,
    64 using the available columns and sorting the way you want it.
    65 
    66 == Ticket columns ==
     56== Creating Custom Reports
     57
     58Creating a custom report requires a comfortable knowledge of SQL.
     59
     60Note that you need grant [TracPermissions#Reports permissions] in order to see the buttons for adding or editing reports.
     61
     62A report is basically a single named SQL query, executed and presented by Trac. Reports can be viewed and created from a custom SQL expression directly in the web interface.
     63
     64Typically, a report consists of a SELECT-expression from the 'ticket' table, using the available columns and sorting the way you want it.
     65
     66== Ticket columns
     67
    6768The ''ticket'' table has the following columns:
    6869 * id
    … …  
    7172 * changetime
    7273 * component
    73  * severity 
    74  * priority 
     74 * severity
     75 * priority
    7576 * owner
    7677 * reporter
    … …  
    8687See TracTickets for a detailed description of the column fields.
    8788
    88 '''all active tickets, sorted by priority and time'''
    89 
    90 '''Example:''' ''All active tickets, sorted by priority and time''
     89Example: '''All active tickets, sorted by priority and time'''
     90{{{#!sql
     91SELECT id AS ticket, status, severity, priority, owner, time AS created, summary
     92FROM ticket
     93WHERE status IN ('new', 'assigned', 'reopened')
     94ORDER BY priority, time
     95}}}
     96
     97== Advanced Reports: Dynamic Variables
     98
     99For more flexible reports, Trac supports the use of ''dynamic variables'' in report SQL statements.
     100In short, dynamic variables are ''special'' strings that are replaced by custom data before query execution. Dynamic variables are entered through the preferences form and the values are autocompleted //(Since 1.3.2)//.
     101
     102=== Using Variables in a Query
     103
     104The syntax for dynamic variables is simple, any upper case word beginning with `$` is considered a variable.
     105
     106Example:
     107{{{#!sql
     108SELECT id AS ticket,summary FROM ticket WHERE priority=$PRIORITY
     109}}}
     110
     111The value of the dynamic variable can be assigned in the report preferences form.
     112
     113To assign a value to `$PRIORITY` in the URL for a report, leave out the leading `$`:
    91114{{{
    92 SELECT id AS ticket, status, severity, priority, owner,
    93        time as created, summary FROM ticket
    94   WHERE status IN ('new', 'assigned', 'reopened')
    95   ORDER BY priority, time
    96 }}}
    97 
    98 
    99 ----
    100 
    101 
    102 == Advanced Reports: Dynamic Variables ==
    103 For more flexible reports, Trac supports the use of ''dynamic variables'' in report SQL statements.
    104 In short, dynamic variables are ''special'' strings that are replaced by custom data before query execution.
    105 
    106 === Using Variables in a Query ===
    107 The syntax for dynamic variables is simple, any upper case word beginning with '$' is considered a variable.
    108 
    109 Example:
     115 https://trac.edgewall.org/reports/14?PRIORITY=high
     116}}}
     117
     118To use multiple variables, separate them with an `&`:
    110119{{{
     120 https://trac.edgewall.org/reports/14?PRIORITY=high&SEVERITY=critical
     121}}}
     122
     123It is possible to assign a default value to the variable, within a SQL comment:
     124
     125{{{#!sql
     126-- PRIORITY = high
     127
    111128SELECT id AS ticket,summary FROM ticket WHERE priority=$PRIORITY
    112129}}}
    113130
    114 To assign a value to $PRIORITY when viewing the report, you must define it as an argument in the report URL, leaving out the leading '$'.
    115 
    116 Example:
    117 {{{
    118  http://trac.edgewall.org/reports/14?PRIORITY=high
    119 }}}
    120 
    121 To use multiple variables, separate them with an '&'.
    122 
    123 Example:
    124 {{{
    125  http://trac.edgewall.org/reports/14?PRIORITY=high&SEVERITY=critical
    126 }}}
    127 
    128 
    129 === Special/Constant Variables ===
    130 There is one ''magic'' dynamic variable to allow practical reports, its value automatically set without having to change the URL.
    131 
    132  * $USER -- Username of logged in user.
    133 
    134 Example (''List all tickets assigned to me''):
    135 {{{
     131
     132=== !Special/Constant Variables
     133
     134There is one dynamic variable whose value is set automatically (the URL does not have to be changed) to allow practical reports.
     135
     136 * $USER — Username of logged in user.
     137
     138Example: List all tickets assigned to me:
     139{{{#!sql
    136140SELECT id AS ticket,summary FROM ticket WHERE owner=$USER
    137141}}}
    138142
    139 
    140 ----
    141 
    142 
    143 == Advanced Reports: Custom Formatting ==
    144 Trac is also capable of more advanced reports, including custom layouts,
    145 result grouping and user-defined CSS styles. To create such reports, we'll use
    146 specialized SQL statements to control the output of the Trac report engine.
    147 
    148 == Special Columns ==
    149 To format reports, TracReports looks for 'magic' column names in the query
    150 result. These 'magic' names are processed and affect the layout and style of the
    151 final report.
    152 
    153 === Automatically formatted columns ===
    154  * '''ticket''' -- Ticket ID number. Becomes a hyperlink to that ticket.
    155  * '''created, modified, date, time''' -- Format cell as a date and/or time.
    156 
    157  * '''description''' -- Ticket description field, parsed through the wiki engine.
     143== Advanced Reports: Custom Formatting
     144
     145Trac is also capable of more advanced reports, including custom layouts, result grouping and user-defined CSS styles. To create such reports, we will use specialized SQL statements to control the output of the Trac report engine.
     146
     147=== Special Columns
     148
     149To format reports, TracReports look for 'magic' column names in the query result. These 'magic' names are processed and affect the layout and style of the final report.
     150
     151=== Automatically formatted columns
     152
     153 * '''ticket''' — Ticket ID number. Becomes a hyperlink to that ticket.
     154 * '''id''' — same as '''ticket''' above when '''realm''' is not set
     155 * '''realm''' — together with '''id''', can be used to create links to other resources than tickets (e.g. a realm of ''wiki'' and an ''id'' to a page name will create a link to that wiki page)
     156   - for some resources, it may be necessary to specify their ''parent'' resources (e.g. for ''changeset'', the ''repos'') and this can be achieved using the '''parent_realm''' and '''parent_id''' columns
     157 * '''created, modified, date, time''' — Format cell as a date and/or time.
     158 * '''description''' — Ticket description field, parsed through the wiki engine.
    158159
    159160'''Example:'''
    160 {{{
    161 SELECT id as ticket, created, status, summary FROM ticket
    162 }}}
    163 
    164 === Custom formatting columns ===
    165 Columns whose names begin and end with 2 underscores (Example: '''`__color__`''') are
     161{{{#!sql
     162SELECT id AS ticket, created, status, summary FROM ticket
     163}}}
     164
     165Those columns can also be defined but marked as hidden, see [#column-syntax below].
     166
     167See [trac:CookBook/Configuration/Reports] for examples of creating reports for realms other than ''ticket''.
     168
     169=== Custom formatting columns
     170
     171Columns whose names begin and end with 2 underscores (e.g. '''`__color__`''') are
    166172assumed to be ''formatting hints'', affecting the appearance of the row.
    167  
    168  * '''`__group__`''' -- Group results based on values in this column. Each group will have its own header and table.
    169  * '''`__color__`''' -- Should be a numeric value ranging from 1 to 5 to select a pre-defined row color. Typically used to color rows by issue priority.
    170 {{{
    171 #!html
    172 <div style="margin-left:7.5em">Defaults:
     173
     174 * '''`__group__`''' — Group results based on values in this column. Each group will have its own header and table.
     175 * '''`__grouplink__`''' — Make the header of each group a link to the specified URL. The URL is taken from the first row of each group.
     176 * '''`__color__`''' — Should be a numeric value ranging from 1 to 5 to select a pre-defined row color. Typically used to color rows by issue priority.
     177 {{{#!html
     178<div style="margin-left:3em">Defaults:
    173179<span style="border: none; color: #333; background: transparent;  font-size: 85%; background: #fdc; border-color: #e88; color: #a22">Color 1</span>
    174180<span style="border: none; color: #333; background: transparent;  font-size: 85%; background: #ffb; border-color: #eea; color: #880">Color 2</span>
    … …  
    178184</div>
    179185}}}
    180  * '''`__style__`''' -- A custom CSS style expression to use for the current row.
    181 
    182 '''Example:''' ''List active tickets, grouped by milestone, colored by priority''
    183 {{{
     186 * '''`__style__`''' — A custom CSS style expression to use on the `<tr>` element of the current row.
     187 * '''`__class__`''' — Zero or more space-separated CSS class names to be set on the `<tr>` element of the current row. These classes are added to the class name derived from `__color__` and the odd / even indicator.
     188
     189'''Example:''' List active tickets, grouped by milestone, group header linked to milestone page, colored by priority:
     190{{{#!sql
    184191SELECT p.value AS __color__,
    185192     t.milestone AS __group__,
     193     '../milestone/' || t.milestone AS __grouplink__,
    186194     (CASE owner WHEN 'daniel' THEN 'font-weight: bold; background: red;' ELSE '' END) AS __style__,
    187        t.id AS ticket, summary
    188   FROM ticket t,enum p
    189   WHERE t.status IN ('new', 'assigned', 'reopened')
    190     AND p.name=t.priority AND p.type='priority'
    191   ORDER BY t.milestone, p.value, t.severity, t.time
    192 }}}
    193 
    194 '''Note:''' A table join is used to match ''ticket'' priorities with their
    195 numeric representation from the ''enum'' table.
    196 
    197 === Changing layout of report rows ===
    198 By default, all columns on each row are display on a single row in the HTML
    199 report, possibly formatted according to the descriptions above. However, it's
    200 also possible to create multi-line report entries.
    201 
    202  * '''`column_`''' -- ''Break row after this''. By appending an underscore ('_') to the column name, the remaining columns will be be continued on a second line.
    203 
    204  * '''`_column_`''' -- ''Full row''. By adding an underscore ('_') both at the beginning and the end of a column name, the data will be shown on a separate row.
    205 
    206  * '''`_column`'''  --  ''Hide data''. Prepending an underscore ('_') to a column name instructs Trac to hide the contents from the HTML output. This is useful for information to be visible only if downloaded in other formats (like CSV or RSS/XML).
    207 
    208 '''Example:''' ''List active tickets, grouped by milestone, colored by priority, with  description and multi-line layout''
    209 
    210 {{{
     195     t.id AS ticket, summary
     196FROM ticket t,enum p
     197WHERE t.status IN ('new', 'assigned', 'reopened')
     198  AND p.name = t.priority AND p.type = 'priority'
     199ORDER BY t.milestone, p.value, t.severity, t.time
     200}}}
     201
     202Note that table join is used to match ''ticket'' priorities with their numeric representation from the ''enum'' table.
     203
     204=== Changing layout of report rows === #column-syntax
     205
     206By default, all columns on each row are display on a single row in the HTML report, possibly formatted according to the descriptions above. However, it is also possible to create multi-line report entries.
     207
     208 * '''`column_`''' — ''Break row after this''. By appending an underscore ('_') to the column name, the remaining columns will be continued on a second line.
     209
     210 * '''`_column_`''' — ''Full row''. By adding an underscore ('_') both at the beginning and the end of a column name, the data will be shown on a separate row.
     211
     212 * '''`_column`''' — ''Hide data''. Prepending an underscore ('_') to a column name instructs Trac to hide the contents from the HTML output. This is useful for information to be visible only if downloaded in other formats (like CSV or RSS/XML).
     213   This can be used to hide any kind of column, even important ones required for identifying the resource, e.g. `id as _id` will hide the '''Id''' column but the link to the ticket will be present.
     214
     215'''Example:''' List active tickets, grouped by milestone, colored by priority, with  description and multi-line layout:
     216
     217{{{#!sql
    211218SELECT p.value AS __color__,
    212219       t.milestone AS __group__,
    213        (CASE owner 
    214           WHEN 'daniel' THEN 'font-weight: bold; background: red;' 
     220       (CASE owner
     221          WHEN 'daniel' THEN 'font-weight: bold; background: red;'
    215222          ELSE '' END) AS __style__,
    216223       t.id AS ticket, summary AS summary_,             -- ## Break line here
    … …  
    219226       description AS _description_,                    -- ## Uses a full row
    220227       changetime AS _changetime, reporter AS _reporter -- ## Hidden from HTML output
    221   FROM ticket t,enum p
    222   WHERE t.status IN ('new', 'assigned', 'reopened')
    223     AND p.name=t.priority AND p.type='priority'
    224   ORDER BY t.milestone, p.value, t.severity, t.time
    225 }}}
    226 
    227 === Reporting on custom fields ===
    228 
    229 If you have added custom fields to your tickets (a feature since v0.8, see TracTicketsCustomFields), you can write a SQL query to cover them. You'll need to make a join on the ticket_custom table, but this isn't especially easy.
    230 
    231 If you have tickets in the database ''before'' you declare the extra fields in trac.ini, there will be no associated data in the ticket_custom table. To get around this, use SQL's "LEFT OUTER JOIN" clauses. See TracIniReportCustomFieldSample for some examples.
    232 
    233 '''Note that you need to set up permissions in order to see the buttons for adding or editing reports.'''
     228FROM ticket t,enum p
     229WHERE t.status IN ('new', 'assigned', 'reopened')
     230  AND p.name = t.priority AND p.type = 'priority'
     231ORDER BY t.milestone, p.value, t.severity, t.time
     232}}}
     233
     234=== Reporting on custom fields
     235
     236If you have added [TracTicketsCustomFields custom fields] to your tickets, you can write a SQL query to include them in a report. You'll need to make a join on the `ticket_custom` table.
     237
     238If you have tickets in the database ''before'' you declare the extra fields in trac.ini, there will be no associated data in the `ticket_custom` table. To get around this, use SQL's `LEFT OUTER JOIN` clauses. See [trac:TracIniReportCustomFieldSample TracIniReportCustomFieldSample] for some examples.
     239
     240=== A note about SQL rewriting #rewriting
     241
     242Beyond the relatively trivial replacement of dynamic variables, the SQL query is also altered in order to support two features of the reports:
     243 1. [#sort-order sorting]
     244 1. pagination: limiting the number of results displayed on each page
     245In order to support the first feature, the sort column is inserted in the `ORDER BY` clause in the first position or in the second position if a `__group__` column is specified (an `ORDER BY` clause is created if needed). In order to support pagination, a `LIMIT ... OFFSET ...` clause is appended.
     246The query might be too complex for the automatic rewrite to work correctly, resulting in an erroneous query. In this case you still have the possibility to control exactly how the rewrite is done by manually inserting the following tokens:
     247 - `@SORT_COLUMN@`, the place where the name of the selected sort column will be inserted,
     248 - `@LIMIT_OFFSET@`, the place where the pagination support clause will be added
     249Note that if you write them after an SQL comment, `--`, you'll effectively disable rewriting if this is what you want!
     250
     251Let's take an example, consider the following SQL query:
     252{{{#!sql
     253-- ## 4: Assigned, Active Tickets by Owner ## --
     254
     255--
     256-- List assigned tickets, group by ticket owner, sorted by priority.
     257--
     258
     259SELECT p.value AS __color__,
     260   owner AS __group__,
     261   id AS ticket, summary, component, milestone, t.type AS type, severity, time AS created,
     262   changetime AS _changetime, description AS _description,
     263   reporter AS _reporter
     264FROM ticket t,enum p
     265WHERE status = 'assigned'
     266  AND p.name = t.priority AND p.type = 'priority'
     267ORDER BY __group__, p.value, severity, time
     268}}}
     269
     270The automatic rewrite will be the following (4 rows per page, page 2, sorted by `component`):
     271{{{#!sql
     272SELECT p.value AS __color__,
     273   owner AS __group__,
     274   id AS ticket, summary, component, milestone, t.type AS type, severity, time AS created,
     275   changetime AS _changetime, description AS _description,
     276   reporter AS _reporter
     277FROM ticket t,enum p
     278WHERE status = 'assigned'
     279  AND p.name = t.priority AND p.type = 'priority'
     280ORDER BY __group__ ASC, `component` ASC,  __group__, p.value, severity, time
     281LIMIT 4 OFFSET 4
     282}}}
     283
     284The equivalent SQL query with the rewrite tokens would have been:
     285{{{#!sql
     286SELECT p.value AS __color__,
     287   owner AS __group__,
     288   id AS ticket, summary, component, milestone, t.type AS type, severity, time AS created,
     289   changetime AS _changetime, description AS _description,
     290   reporter AS _reporter
     291FROM ticket t,enum p
     292WHERE status = 'assigned'
     293  AND p.name = t.priority AND p.type = 'priority'
     294ORDER BY __group__, @SORT_COLUMN@, p.value, severity, time
     295@LIMIT_OFFSET@
     296}}}
     297
     298If you want to always sort first by priority and only then by the user selected sort column, simply use the following `ORDER BY` clause:
     299{{{#!sql
     300ORDER BY __group__, p.value, @SORT_COLUMN@, severity, time
     301}}}
     302
     303== Changing Report Numbering
     304
     305There may be instances where you need to change the ID of the report, perhaps to organize the reports better. At present this requires changes to the trac database. The ''report'' table has the following schema:
     306 * id integer PRIMARY KEY
     307 * author text
     308 * title text
     309 * query text
     310 * description text
     311Changing the ID changes the shown order and number in the ''Available Reports'' list and the report's perma-link. This is done by running something like:
     312{{{#!sql
     313UPDATE report SET id = 5 WHERE id = 3;
     314}}}
     315Keep in mind that the integrity has to be maintained, i.e. ID has to be unique, and you don't want to exceed the max for your database.
     316
     317You may also need to update or remove the report number stored in the report or query.
    234318
    235319----
    236 See also: TracTickets, TracQuery, TracGuide, [http://www.sqlite.org/lang_expr.html Query Language Understood by SQLite]
     320See also: TracTickets, TracQuery, [https://www.sqlite.org/lang_expr.html Query Language Understood by SQLite]