A technical journal and community hub from Redgate. It is an interesting problem in Transact SQL, for which there are a number of solutions and considerable debate. How do you go about producing a summary result in which a distinguishing column from each row in each particular category is listed in a 'aggregate' column?
A simple, and intuitive way of displaying data is surprisingly difficult to achieve. Anith Sen gives a summary of different ways, and offers words of caution over the one you choose. Many a time, SQL programmers are faced with a requirement to generate report-like resultsets directly from a Transact SQL query. In most cases, the requirement arises from the fact that there neither sufficient tools nor in-house expertise to develop tools that can extract the data as a resultset, and then massage the data in the desired display format.
Quite often folks are confused about the potential of breaking relational fundamentals such as the First Normal Form or the scalar nature of typed values. Talking about 1NF violations in a language like SQL which lacks sufficient domain support, allows NULL s and supports duplicates is somewhat ironic to begin with, but that is a topic which requires detailed explanations.
You have a table, view or result that looks like this… …and you wish to have a resultset like the one below:. The objective is to return a resultset with two columns, one with the Category Identifier, and the other with a concatenated list of all the Product Names separated by a delimiting character: Concatenating column values or expressions from multiple rows are usually best done in a client side application language, since the string manipulation capabilities of Transact SQL and SQL based DBMSs are somewhat limited.
However, you can do these using different approaches in Transact SQL, but it is best to avoid such methods in long-term solutions.
Even though SQL, in general, deviates considerably from the relational model, its reliance on certain core aspects of relational foundations makes SQL functional and powerful.
One such core aspect is the set based nature of SQL expressions well, multi-sets to be exact, but for the given context let us ignore the issue of duplication. The primary idea is that tables are unordered and therefore the resultsets of any query that does not have an explicit ORDER BY clause is unordered as well. In other words, the rows in a resultset of a query do not have a prescribed position, unless it is explicitly specified in the query expression. On the other hand, a concatenated list is an ordered structure.
Each element in the list has a specific position. In fact, concatenation itself is an order-utilizing operation in the sense that values can be prefixed or post fixed to an existing list.
If such an ordering criteria is not provided, the concatenated string would be arbitrary in nature. Generally, requests for row value concatenations often comes in two basic flavors, when the number of rows is known and small typically less than 10 and when the number of rows is unknown and potentially large. It may be better to look at each of them separately. In some cases, all the programmer wants is just the list of values from a set of rows.
In such situations, the approaches can be the same except that the join conditions may vary. Minor variations of the examples list on this page illustrate such solutions as well. For the purpose of this article the Products table from Northwind database is used to illustrate column value concatenations with a grouping column. Northwind is a sample database in SQL Server default installations.
You can download a copy from from the Microsoft Downloads. When the number of rows is small and almost known beforehand, it is easier to generate the code. One common approach where there is a small set of finite rows is the pivoting method. Here is an example where only the first four alphabetically-sorted product names per categoryid is retrieved:. The idea here is to create a expression inside the correlated subquery that produces a rank seq based on the product names and then use it in the outer query.
If you are using any previous version, you will have to use the subquery approach You can also use a self-join, to write it a bit differently. Using the recently introduced PIVOT operator, you can write this as follows:. Not only does the syntax appear a bit confusing, but also it does not seem to offer any more functionality than the previous CASE approach.
However, in rare situations, it could come in handy. The new features in SQL make some of the approaches easier. The examples below make this point obvious. The idea behind this method is from a newsgroup posting by Vadim Tropashko. It is similar to the ideas behind generating a materialized path for hierarchies.
The CASE in the recursive part of the CTE is used to eliminate the initial comma, but you can use RIGHT or the SUBSTRING functions instead. This may not be the best performing option, but certain additional tuning could be done to make them suitable for medium sized datasets. Another approach using recursive common table expressions was sent in by Anub Philip, an Engineer from Sathyam Computers that uses separate common table expressions for the anchor and recursive parts.
On first glance, this query may seem a bit expensive in comparison, but the reader is encouraged to check the execution plans and make any additional tweaks as needed. Here is a technique for string concatenation that uses the FOR XML clause with PATH mode. It was initially posted by Eugene Kogan, and later became common in public newsgroups.
There is a similar approach that was originally found in the beta newsgroups, using the CROSS APPLY operator. You may notice a comma at the end of the concatenated string, which you can remove using a STUFF, SUBSTRING or LEFT function. While the above methods are deemed reliable by many at the time of writing, there is no guarantee that it will stay that way, given that the internal workings and evaluation rules of FOR XML PATH expression in correlated subqueries are not well documented.
Though this article is about approaches using Transact SQL, this section is included due to the popularity of CLR aggregates in SQL It not only empowers the CLR programmer with new options for database development, but also, in some cases, they work at least as well as native Transact SQL approaches. If you are familiar with. NET languages, SQL offers a convenient way to create user defined aggregate functions using CVB. NET or similar languages that are supported by the Common Language Runtime CLR.
Here is an example of a string concatenate aggregate function written using C. Once you build and deploy this assembly on the server, you should be able to execute your concatenation query as:. If you are a total newbie on CLR languages, and would like to learn more about developing database solutions using CLR languages, consider starting at Introduction to Common Language Runtime CLR Integration.
Recursive functions in t-SQL have a drawback that the maximum nesting level is So this approach is applicable only for smaller datasets, especially when the number of items within a group, that needs to be concatenated, is less than This approach is based on the idea by Linda Wierzbecki where a table variable with three columns is used within a table-valued UDF.
The first column represents the group, second represents the currently processing value within a group and the third represents the concatenated list of values.
There is enough literature out there which demonstrates the drawbacks and implications of using Dynamic SQL. The drawbacks of rampant usage of cursors are well-known among the Transact SQL community. Because they are generally resource intensive, procedural and inefficient, one should strive to avoid cursors or loop based solutions in general Transact SQL programming. This section details a couple of notorious methods often publicized by some in public forums.
The problem with these methods is that they rely on the physical implementation model; changes in indexes, statistics etc or even a change of a simple expression in the SELECT list or ORDER BY clause can change the output.
Also these are undocumented, unsupported and unreliable to the point where one can consistently o que quer dizer stock options failures. Therefore these methods are not recommended at all for production mode systems.
It pound exchange rate in pakistani rupees rare for the usage of an expression that involves a column, a variable and an expression in the SET clause in an UPDATE statement to appear intuitive. However, in general, the optimizer often seems to process these values in the order of materialization, either in the internal work tables or any other storage structures. Again, it is important to consider that lack of physical independence that is being exploited here before using or recommending this as a usable and meaningful solution.
This is an approach purely dependent on the physical implementation grants for college students with learning disabilities internal access paths. Before using this approach, make sure to refer to the relevant knowledgebase article. The most logical choice would to have a built-in operator with optional configurable parameters that can do the concatenation of option trading simulator dvd values depending on the type.
Till then, reporting requirements and external data export routines will have to rely on such Transact SQL programming hacks. Umachandar Jayachandran, Linda Wierzbecki, Bruce Margolin, Roy Harvey, Eugene Kogan, Vadim Tropashko, Anub Philip.
Fortnightly newsletters help sharpen your skills and keep you ahead, with articles, ebooks and opinion to keep you informed.
Anith S Larson specializes in data management primarily using SQL Server. From the mid 90s, he has been working on a variety of database design and systems development projects for clients primarily in wealth management and financial services industry. He resides in Lakeland, TN. I use the following. This is not completely my code.
I found most if not nearly all of this on the net. DECLARE ParameterKey INT EXEC dbo. This stock market 2016 2016 vs 2016.5 fun Here is my method.
It works in stock market guru rules SQL Server and DECLARE accumulation VARCHAR—variable used for accumulating lists CategoryID INT —variable used for keeping tabs on the GROUPING id. Transact sql stored procedure parameter default value grouping —and update the table, doing the accumulation.
I suspect that FOR XML PATH will still deliver better performance in many cases, though. I never completely understood the underlying evaluation scheme in t-SQL UPDATE EXTENSION. Is it right to left, all at once, random, I have no idea. Regardless, it transact sql stored procedure parameter default value to work fine here. Another concern is the INSERT. SELECT statement with an ORDER BY clause.
There has been several online discussions for instance, http: That is an excellent point. Rather than australian binary options law in uk REPLACE function, it is better to have the value method of the xml type. I am not completed sure how to proceed when there is a grouping column; Do you include the grouped column as an element in your xml value like: Regarding cursors, yet another option is a CLR stored procedure.
Specifying a Parameter Default Value
You can read the data with SqlDataReader, do the concatenation with StringBuilder, and send all of the results back to the caller in a single rowset rather than stuffing them into a temp table. Much, much more efficient than a T-SQL cursor.
This method works in SQL Server upwards. Options futures and other derivatives ebook far as I know, Robyn and I invented it, though it took three cups of coffee. Excellent Article Excellent article and it had some techniques I had never considered before. Jeff Moden approached the same topic from a purely performance standpoint in: And some of the CTE portion is similar to http: Here is another way of doing it That is a new one, Phil.
And a good one: This can vary from one release of SQL Server to another. In the case you mention, there is no supported order for the concatenation to occur. The use of an expression in the ORDER BY forces a different execution plan which produces an indeterminate order for the concatenation, presumably because the order-by is postponed until after the concatenation is performed. My own experience is that a simple SQL query will result in a correct concatenation.
If you pile in with the fancy stuff, you run a risk that the concatenation ends up with merely the last string added to it. Splitting a comma-separated-values is a common task I think. Hopefully it will become less common with table valued parameters. Join and split http: Recursion and row pairing declare t table CatID tinyint identity 1,1 primary keyCategoryID tinyint not nullProductName varchar not null.
ProductName from t l join t r on l. Recursion and row pairing correction I apologize for an oversight. The delete statment case was unnecessary. It seems to me a big pro. The resulting query have far more grater readability, and avoid the overhead money makers groupsonline by the other methods. Shamas saeed Considration This is good article. You can show row values in one column usingseperated or any other character.
We can use coleasce function to do this as. I am not familiar with the performance factors related to CLR. Perhaps, someone else may chime in. You can find several examples of failure discussed in the archives of google groups.
As we can see from your examples, with UDF I can encapsulate concatenation logic for a specific column of a specific table, so if I need to concatenate another column from another table I have to write another UDF. Do you think its a good argument to prefer CLR aggregate on other options?
Products order by CategoryId,ProductName. Any idea how to work around this? Anonymous Commenting Disabled Anonymous commenting has been disabled in this article due to spamming. Sorry for the inconvenience. Years before, I might add, people began having heart murmurs over this technique.
It is a good policy not to release code that relies on an unsupported behavior.
I do not think it would be easy for Microsoft to break that particular behavior — they would have to change or take away altogether the ability to set a VARCHAR local variable with a Book currency forex guest trading, and physically do something to keep the implied loop in the SELECT from running. Thanks for a good article. CLR Error Great artical Anith!! I really liked the idea of using CLR to add this function so it could be reused easily.
Because I am very new to C I do not know how to fix this issue, and thought I would notify you that your sample may have a cakephp form input select selected. Shamas saeed Considration Like Lee, I use this method a lot. I accept that learn stock market basics pdf order of items in the concatenated list is not reliable.
Actually, variable concatenation in SELECT is Reliable For SQL Server at least, it is not correct to say that variable concatenation in SELECT is either unreliable or dependent on the physical implementation. The example given is unreliable because it has an error in that it lacks an ORDER BY clause to insure the proper ordering.
Figure Out the Default Values of Stored Procedure Parameters - CodeProject
Here is a correct implementation:. This is all documented and supported straight out of BOL. Finally, the referenced knowledgebase article is not relevant because it is only for SQL Server and 7. There are several cases where it fails. Several forum members in Microsoft newsgroups stock market in riyadh posted several examples, at least from version 7.
For some examples, see:. The problem is several people ignore these warnings and keep using them only to find one day it suddenly stopped working! This code is not entirely complete, since it will fall over when there are no rows in the product table, but you get the idea.
Much simpler way part 2 Oops — just realised that I was solving a different problem — mine only works for a single group at a time… sorry for confusion. How do I add new lines instead of commas?
I am trying to concatenate row values with newlines in stead. I need to display the query results on a ColdFusion 8 website. I love this article problem is I dont understand it completely. Can someone please help me edit this process to wrok in MSAccess? I have the following data with multiple City and state Values that program for free binary options signals providers equal but unique zipcodes.
This list is 43, rows long:. Meaning if there are 3 or 50 Raleigh NC in the table it will make one and add all the zip codes to the one cell with the city state name.
I would appreciatte any help in advance I have been trying to figure out binary option south africa to do this in excel or access for about 2 weeks now.
No, you are not missing anything. Great examples Thanks for the examples of the various methods. This gives me more choices in how to solve the concatenation problem. Products FOR XML PATH.
Migrating from Oracle to SQL Server — Aggregate function Hi, Need help in converting the StrAgg ODCIAggregate from Oracle function http: There are many Views which use this function in generic way. Protocol From PAMSTR T, CITY Cy Where Cy. CycleNum P, PAMSTRDTL Pl Where P. If I execute without Aggregate I get: Docid Protocol Jan Feb Mar 1 P1 Jan 1 P1 Jan Mar 2 P2 Mar 2 P2 Jan Mar.
If I execute with Aggregate I get: Docid Protocol Jan Feb Mar 1 P1 Jan, Jan Mar 2 P2 Jan Mar, Mar. Peeling back Great article, it solved my issue, I would like to say thank to all of you, regarding to Anith Sen. But sql server prevents me from creating an index on views that use CTE and subquery. Can you create new record Hi, i like to know how to create new record after the fourth records?
Using the CLR routine? ASA has no trouble referring to aliases in WHERE clauses. What if we have another column. Here only two columns are there. Visit our library of articles to discover the patterns and practices you need to move towards more agile methods of database delivery.
Find out how to automate the process of building, testing and deploying your database changes to reduce risk and speed up the delivery cycle. What do you think of the new Simple Talk?
Give us your feedback. NET Cloud Sysadmin Opinion Books Blogs. Home SQL T-SQL Programming Concatenating Row Values in Transact-SQL 31 July Concatenating Row Values in Transact-SQL It is an interesting problem in Transact SQL, for which there are a number of solutions and considerable debate. Considerations Concatenating values when the number of items is small and known beforehand Concatenating values when the number of items is not known.
Recursive CTE methods The blackbox XML methods Using Common Language Runtime. Scalar UDF with recursion. Table valued UDF with a WHILE loop. Dynamic SQL The Cursor approach.
Unreliable approaches Scalar UDF with t-SQL update extension. Scalar UDF with variable concatenation in SELECT Conclusion. References Acknowledgements Introduction Many a time, SQL programmers are faced with a requirement to generate report-like resultsets directly from a Transact SQL query. You have a table, view or result that looks like this… …and you wish to have a resultset like the one below: In this example we are accessing the sample NorthWind database and using the following SQL.
SELECT CategoryIdProductName. MAX CASE seq WHEN 4 THEN ProductName ELSE '' END. Products p1 D CategoryIdProductNameseq. GROUP BY CategoryId. FROM SELECT CategoryIdProductName. Products P CategoryIdProductNameseq. AS SELECT CategoryIdCAST '' AS VARCHARCAST '' AS VARCHAR0. RANK OVER PARTITION BY CategoryId ORDER BY length DESC. WITH Ranked CategoryIdrnkProductName. AS SELECT CategoryId. CAST ProductName AS VARCHAR AnchorRanked CategoryIdrnkProductName.
AS SELECT CategoryIdrnkProductName. RecurRanked CategoryIdrnkProductName. SELECT CategoryIdMAX ProductName. FOR XML PATH '' AS Products. SELECT DISTINCT CategoryIdProductNames. FOR XML PATH ''TYPE. Join ""this. RETURNS VARCHAR AS BEGIN. DECLARE r VARCHARl VARCHAR Product VARCHAR 40. INSERT t CategoryIdProductlist. Product''. FROM t t END. SELECT CategoryIdlist AS Products. DECLARE r VARCHAR MAXn INTi INT.
ORDER BY COUNT ProductName DESC. FROM SELECT CategoryId, ProductName. Products p D CategoryId, ProductName, Seq.
DECLARE tbl TABLE id INT PRIMARY KEYlist VARCHAR DECLARE c INTp VARCHARcNext INTpNext VARCHAR DECLARE c CURSOR FOR. ORDER BY CategoryIdProductName. FETCH NEXT FROM c INTO cNextpNext. INSERT tbl SELECT cp. FETCH NEXT FROM c INTO cNextpNext. RETURNS VARCHAR MAX AS. DECLARE t TABLE p VARCHAR 40.
DECLARE r VARCHAR MAX. INSERT t p SELECT ProductName FROM Northwind. SELECT CategoryIddbo. RETURNS VARCHAR MAX AS BEGIN. DECLARE p VARCHAR MAX. Subscribe for more articles Fortnightly newsletters help sharpen your skills and keep you ahead, with articles, ebooks and opinion to keep you informed. Subscribe to our fortnightly newsletter. Anith Sen Anith S Larson specializes in data management primarily using SQL Server. View all articles by Anith Sen.
My Solution I use the following. CLR methods Nice to see you here, Anith! This looks fun Good job, Phil. CLR methods Hello Adam, That is an excellent point. Good article This is a useful article.
Some methods are very new to me. Great article Recursive CTE methods is bit hard to understand. Here is another way of doing it This method works in SQL Server upwards.
Anith, your article was extremely thorough and enlightening. Question, what happens if I change the ORDER BY clause as the following and why? ORDER BY LEFT categoryID, 5productName. Excellent Article Thanks for the links, Timothy.
Thanks for a fascinating article. It has certainly taught me several things. Excellent article This is an excellent article.
Thanks I Apologize for my poor English… ;P. Perhaps, someone else may chime in, Anith. Good one I have learnt lot of things from this artical. Good, now go try to solve a T-SQL string concatenation challenge and win a prize: Here is a correct implementation: Variable concatenation in SELECT RBarryYoung and Dennis, There are several cases where it fails. For some examples, see: Much Simpler Way Hi Anith This is how I do it.
Not sure why nobody else sems to do it this way: I use this approach to build values for variable PIVOT clauses. The pre-ranking of the records for the CTE was a huge benefit.
The slower method did run quite fast on a simpler data set, but sank on the more complex table. This list is 43, rows long: Why not this solution??
Concat using XQuery Hi guys! Products FOR XML PATH ; select iXml. Docid Protocol Jan Feb Mar 1 P1 Jan 1 P1 Jan Mar 2 P2 Mar 2 P2 Jan Mar If I execute with Aggregate I get: Docid Protocol Jan Feb Mar 1 P1 Jan, Jan Mar 2 P2 Jan Mar, Mar Any guidance appreciated to get generic solution.
SQL Server Performance SQL Server Performance Tuning for Stored Procedures
Insane Fair dinkum, MSSQL drives me mental. Sybase Adaptiver Server Anywhere ASA has an aggregate list function. SELECT CategoryId, list ProductName FROM Northwind. Products GROUP BY CategoryId How hard is that? Also in SQL Relational Algebra and its implications for NoSQL databases With the rise of NoSQL databases that are exploiting aspects of SQL for querying, and are embracing full transactionality, is there a danger of the data-document model's hierarchical nature causing a fundamental conflict with relational theory?
We asked our relational expert, Hugh Bin-Haad to expound a difficult area for database theorists. Also in T-SQL T-SQL Window Function Speed Phreakery: The FIFO Stock Inventory Problem Sometimes, in the quest for raw SQL performance, you are forced to sacrifice legibility and maintainability of your code, unless you then document your code lavishly.
Phil Factor's SQL Speed Phreak challenge produced some memorable code, but can SQL features introduced since then help to produce code that performs as well and is also easy to understand? Also in T-SQL Programming SQL Server System Functions: The Basics Every SQL Server Database programmer needs to be familiar with the System Functions. These range from the sublime such as rowcount or identity to the ridiculous IsNumeric Robert Sheldon provides an overview of the most commonly used of them.
Also in T-SQL Programming Database Code Analysis Database code analysis will reduce the number of 'code smells' that creep into your database builds. It will alert the team to mistakes or omissions, such as missing indexes, that are likely to cause performance problems in production.
It will allow the Governance and Operations team visibility into production readiness of the code, warning them of security loopholes and vulnerabilities. William Brewer describes the two technical approaches to database code analysis, static and dynamic, and suggests some tools that can help you get started.
Generic ; using System. SqlTypes ; using System. IO ; using Microsoft.SQL Server - INSERT RECORDS INTO TABLE VIA STORED PROCEDURE AND DEFAULT VALUES