{"id":334,"date":"2012-10-22T09:55:03","date_gmt":"2012-10-22T16:55:03","guid":{"rendered":"http:\/\/www.sqldbpros.com\/?p=334"},"modified":"2012-11-14T12:49:52","modified_gmt":"2012-11-14T19:49:52","slug":"sql-script-distinct-column-values-across-all-tables","status":"publish","type":"post","link":"http:\/\/sqldbpros.com\/wordpress\/2012\/10\/sql-script-distinct-column-values-across-all-tables\/","title":{"rendered":"SQL Script: Distinct Column Values Across All Tables"},"content":{"rendered":"<p>Sometimes I wonder if some of the more random scripts I post are useful to anyone but me so if this one is useful to you definitely drop a comment below!<\/p>\n<p>Check it out:\u00a0 Our team was trying to confirm what our best practice was when creating flag columns in the data warehouse.\u00a0 As usual someone made an appeal to history: \"What have we done in the past?\" (Note: this isn't really a valid way to establish a best practice but whatever Heather. <em>Yes! Movie tie in! Bonus points to myself for it being an 80's movie.\u00a0 Heathers! Pic and link below).<\/em><\/p>\n<p>Back to the story:\u00a0 What values do we typically use for flag columns in the data warehouse? 0 or 1? Y or N? Yes or No? T or F? True or False?\u00a0 What have we done in the past?\u00a0 (<em>Another note: It's the data warehouse.\u00a0 The goal is to make it easy for the users.\u00a0 This eliminates 0\/1. I'd also eliminate Y\/N. Personal preference: True\/False<\/em>).<\/p>\n<p>With a little information_schema hackery we can generate a nice little SQL script to get the answer for us.\u00a0 The basis is something like this:<!--more--><\/p>\n<pre>SELECT  c.TABLE_SCHEMA\r\n      , c.TABLE_NAME\r\n      , c.COLUMN_NAME\r\n      , p.rows\r\n      , 'select ''' + c.TABLE_SCHEMA + ''' as SchemaName, ''' + c.TABLE_NAME\r\n        + ''' as TableName, ''' + c.COLUMN_NAME\r\n        + ''' as ColumnName, count(1) as RowCnt, convert(varchar(200), '\r\n        + c.COLUMN_NAME + ') as ExistingValues from ' + c.TABLE_SCHEMA + '.'\r\n        + c.TABLE_NAME + ' group by ' + c.COLUMN_NAME + ' UNION ALL'\r\nFROM    INFORMATION_SCHEMA.COLUMNS AS c\r\n        JOIN INFORMATION_SCHEMA.tables AS t\r\n            ON c.TABLE_CATALOG = t.TABLE_CATALOG\r\n               AND c.TABLE_SCHEMA = t.TABLE_SCHEMA\r\n               AND c.TABLE_NAME = t.TABLE_NAME\r\n        JOIN sys.partitions AS p\r\n            ON c.TABLE_SCHEMA = OBJECT_SCHEMA_NAME(p.object_id)\r\n               AND c.TABLE_NAME = OBJECT_NAME(p.object_id)\r\n               AND p.index_id &lt; 2\r\nWHERE  t.TABLE_TYPE = 'base table' \r\n        AND COLUMN_NAME LIKE '%flag%'\r\nORDER BY p.rows DESC<\/pre>\n<p>This script will run fairly quickly.\u00a0 A few seconds or so.\u00a0 The last column of the results will contain the SQL script you need to run to get the actual values.<\/p>\n<p><em>WARNING! Now here's where you need to be cautious!\u00a0 Review the results of the query above.\u00a0 Take a look at the row counts and consider whether doing an aggregation against some of your biggest tables is really necessary (or resource efficient).\u00a0 Customize the script as your needs dictate.<br \/>\n<\/em><\/p>\n<p>Okay, cut and paste the select statement from the original results into a new window.\u00a0 It will looks like this (sorry you'll have to format your results yourself):<\/p>\n<pre>SELECT  'dbo' AS SchemaName\r\n      , 'TableA' AS TableName\r\n      , 'LikesHeathersFlag' AS ColumnName\r\n      , COUNT(1) AS RowCnt\r\n      , CONVERT(VARCHAR(200), LikesHeathersFlag) AS ExistingValues\r\nFROM    dbo.TableA\r\nGROUP BY LikesHeathersFlag\r\nUNION ALL\r\nSELECT  'dbo' AS SchemaName\r\n      , 'TableB' AS TableName\r\n      , 'CancelFlag' AS ColumnName\r\n      , COUNT(1) AS RowCnt\r\n      , CONVERT(VARCHAR(200), CancelFlag) AS ExistingValues\r\nFROM    dbo.TableB\r\nGROUP BY CancelFlag\r\nUNION ALL\r\nSELECT  'dbo' AS SchemaName\r\n      , 'TableC' AS TableName\r\n      , 'AnotherFlag' AS ColumnName\r\n      , COUNT(1) AS RowCnt\r\n      , CONVERT(VARCHAR(200), AnotherFlag) AS ExistingValues\r\nFROM    dbo.TableC\r\nGROUP BY AnotherFlag\r\n<span style=\"background: #00ff00;\">UNION ALL<\/span><\/pre>\n<p>If you run this as is you'll get a syntax error.\u00a0 Scroll to the bottom of the query and remove the final \"UNION ALL\" (as well as any tables\/columns you don't want to include in the results).<\/p>\n<p>In my case I also added a temp table so I could do faster analysis of the results.\u00a0 My final code looked like this:<\/p>\n<pre>SELECT  'dbo' AS SchemaName\r\n      , 'TableA' AS TableName\r\n      , 'LikesHeathersFlag' AS ColumnName\r\n      , COUNT(1) AS RowCnt\r\n      , CONVERT(VARCHAR(200), LikesHeathersFlag) AS ExistingValues\r\n<span style=\"background: #00ff00;\">INTO #FlagValues<\/span>\r\nFROM    dbo.TableA\r\nGROUP BY LikesHeathersFlag\r\nUNION ALL\r\nSELECT  'dbo' AS SchemaName\r\n      , 'TableB' AS TableName\r\n      , 'CancelFlag' AS ColumnName\r\n      , COUNT(1) AS RowCnt\r\n      , CONVERT(VARCHAR(200), CancelFlag) AS ExistingValues\r\nFROM    dbo.TableB\r\nGROUP BY CancelFlag\r\nUNION ALL\r\nSELECT  'dbo' AS SchemaName\r\n      , 'TableC' AS TableName\r\n      , 'AnotherFlag' AS ColumnName\r\n      , COUNT(1) AS RowCnt\r\n      , CONVERT(VARCHAR(200), AnotherFlag) AS ExistingValues\r\nFROM    dbo.TableC\r\nGROUP BY AnotherFlag<\/pre>\n<p>&nbsp;<\/p>\n<figure id=\"attachment_336\" aria-describedby=\"caption-attachment-336\" style=\"width: 540px\" class=\"wp-caption aligncenter\"><a href=\"http:\/\/www.amazon.com\/gp\/product\/B001DHXSXK\/ref=as_li_qf_sp_asin_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=B001DHXSXK&amp;linkCode=as2&amp;tag=grocwine-20\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-336\" title=\"allvaluesforacolumnacrossdatabase\" src=\"http:\/\/www.sqldbpros.com\/wordpress\/wp-content\/uploads\/2012\/10\/allvaluesforacolumnacrossdatabase.jpg\" alt=\"\" width=\"540\" height=\"279\" srcset=\"http:\/\/sqldbpros.com\/wordpress\/wp-content\/uploads\/2012\/10\/allvaluesforacolumnacrossdatabase.jpg 540w, http:\/\/sqldbpros.com\/wordpress\/wp-content\/uploads\/2012\/10\/allvaluesforacolumnacrossdatabase-300x155.jpg 300w\" sizes=\"auto, (max-width: 540px) 100vw, 540px\" \/><\/a><figcaption id=\"caption-attachment-336\" class=\"wp-caption-text\">Really? A zero or one in a data warehouse flag column? You must be joking.<\/figcaption><\/figure>\n<p><a href=\"http:\/\/www.amazon.com\/gp\/product\/B001DHXSXK\/ref=as_li_qf_sp_asin_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=B001DHXSXK&amp;linkCode=as2&amp;tag=grocwine-20\">Heathers<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Sometimes I wonder if some of the more random scripts I post are useful to anyone but me so if this one is useful to you definitely drop a comment below! Check it out:\u00a0 Our team was trying to confirm what our best practice was when creating flag columns in the data warehouse.\u00a0 As usual [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[22,17,43,3,57,72],"tags":[35,83,20,81,32,34,82,80,10,6,9,12,11,40,173],"class_list":["post-334","post","type-post","status-publish","format-standard","hentry","category-businss-intelligence","category-data-architect","category-microsoft-sql-server","category-scripts","category-t-sql","category-table-design","tag-best-practice","tag-bit","tag-business-intelligence","tag-data-warehouse","tag-design-review","tag-development","tag-flag","tag-heathers","tag-ms-sql","tag-script","tag-sql-server","tag-sql-server-2005","tag-sql-server-2008","tag-standards","tag-t-sql"],"_links":{"self":[{"href":"http:\/\/sqldbpros.com\/wordpress\/wp-json\/wp\/v2\/posts\/334","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/sqldbpros.com\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/sqldbpros.com\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/sqldbpros.com\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/sqldbpros.com\/wordpress\/wp-json\/wp\/v2\/comments?post=334"}],"version-history":[{"count":6,"href":"http:\/\/sqldbpros.com\/wordpress\/wp-json\/wp\/v2\/posts\/334\/revisions"}],"predecessor-version":[{"id":408,"href":"http:\/\/sqldbpros.com\/wordpress\/wp-json\/wp\/v2\/posts\/334\/revisions\/408"}],"wp:attachment":[{"href":"http:\/\/sqldbpros.com\/wordpress\/wp-json\/wp\/v2\/media?parent=334"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/sqldbpros.com\/wordpress\/wp-json\/wp\/v2\/categories?post=334"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/sqldbpros.com\/wordpress\/wp-json\/wp\/v2\/tags?post=334"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}