blah

blah

<div>content</div>

<div>content More content</div>

blah

blah

Oops forgot to close the previous Div

blah

blah

<tr><td><td></tr>

1. The attribute they are looking for will always be the first
2. The order of the attribute are fixed
3. That the attributes are always quoted
4. All attributes assign a value.

blah

blah

</p>
<p class=”can you findMe”> More stuff</p>
<p> More stuff</p>

Trying to find the find each element with findMe in the class with a regex is way more trouble than it’s worth.

Now while these example are contrived and you most likely won’t get such a mix of styles.  You could get one that you weren’t expecting.  And if you HTML was created by a team of developer you may get that mix after all.  And remember you can’t even assume the HTML is correctly written. So the moment you notice you’ve missed a style you didn’t account for you have to adjust your pattern which could be a huge nightmare.

Another thing to consider especially when searching by attributes is the a regex can’t reuse consumed text. Consider the following
<div class=”findMe”>
    <p>Blah</p>
    <p class=”findMe”> Some text <b class=”findMe”>to find</b></p>
    <p>Blah</p>
</div>
<div> Blah Blah Blah</div>

Again if the task is to find all elements with a find me class. Let’s assume a regex can match the div element.  Once this is done the regex engine will proceed to find the next match after the div element’s closing tags.  It will not go back inside the div to find the class on the elements within the div.


Instead of trying to hit a moving target with a regex, the HTML DOM would parse all the above samples, even the incorrect one, because that's it job.   Of course if there are errors the document may not be parse exactly as you intended but it will be correctly parsed. From there it is fairly easy to find the element you need. Reformatting the markup is very unlikely to break code using the the DOM.

For example using the popular JavaScript library JQuery the follow snippet would find all all the elements with a findMe class.

var foundThem = $(".findMe");

Or in the versions of JavaScript supporting getElementByClassName

var foundThem = document.getElementsByClassName('findMe');

And quickly XML which has stricter syntax rules, well formed, all attributes quoted, may seem more regex friendly but there are other ways to parse XML.  Xpath or XQuery both more maintainable than a regex would be for this task and easier to use and understand in most cases.

You can use a regex which may work, some of the time, or the DOM, something that should work all the time.  In the end the choice is yours.

The first pattern had three conditions that were being tested for with lookaheads.

^(?=.*\d)(?=.*[a-z])(?=.*[A-Z]).{8,15}$

With the current pattern only one lookahead was being used.

^(?=.*?\d)[a-z][a-z0-9]{5,7}$

In both patterns the pattern had a min and max length. In to original attempts of both the length was being checked after the lookahead(s) test. While this is perfectly fine in a non VBScript/JScript world, this is were the bug kicks in those regex engines. Actually it's probably the same regex engine for both languages, which is probably why it only effects IE it's the only browser that uses VBScript and Jscript natively. I don't recall in my original testing for this bug if I tested it server-side given my previous blog comments, mostly likely I only tested client-side. However the recent question it was failing server-side so it's not actually in the browser but more likely the DLL for those languages. Anyway the previous blog article covered the behavior that was happening.

Steve Levithan looked much closer at the problem in general and discussed it on his blog. He came to the conclusion that the qualifiers with a minimum boundary of zero, within the lookahead were the culprit. He provides a couple of simple examples. I think he's partially right but I don't think the 1+ qualifiers are excluded from the problem. I think his provided examples were a little too simple for them to be effected.

OK let look at the regex pattern in the recent question

^(?=.*?\d)[a-z][a-z0-9]{5,7}$

The requirements were 6 to 8 alphanumeric (English) string that started with an alpha character, with at least one digit. Let's use “abc123” as the text string.

Now the above pattern was supplied by one of the resident pros who frequent the message boards. Let's get pass the fact the pattern itself is correct and satisfies the requirements. I'm going to rewrite the pattern to

^(?=\D*\d)[a-z][a-z0-9]{5,7}$

Now this is functionally the same, it's just within the lookahead the pattern is greedy instead of lazy and it will make the point I'm trying to make easier to see (I hope) without having to deal with backtracking. Now this pattern suffers from the bug in VBScript/JScript. Now Steven suggested the one or more qualifier (+) doesn't suffer from the problem but change the to a + , which doesn't effect the match because the first test after the lookahead is for an alpha so the lookahead placed as it is will always match at least one character with the \D* pattern. So now we have

^(?=\D+\d)[a-z][a-z0-9]{5,7}$

which is also bitten by the bug.

I first tried the same approach on my original attempt dealing with the bug, but no luck. I tried using the + qualifier, still didn't work. Now the person posting the question stated without the lookahead the remaining pattern worked fine, excluding the at least one digit test. So I begin testing the part of patterns such as

^a-z][a-z0-9]{5,7}$

^(?=\D*\d)[a-z]

^(?=\D+\d)[a-z]

are just a few attempts, all work as expected. It was only when I put the whole pattern back together that it begin failing. But after some more trial and error I think I've come across a pattern with the bug. Going back to my original examination of the problem I discovered that this pattern ^(?=\D*\d)[a-z][a-z0-9]{2,7}$ matches the test string “abc123” now this doesn't seem to quite fit with my original assessment of what was happening because in that pattern after the lookahead test it's just testing for a certain number of any characters, but it this pattern it's looking for a specific range of characters but if you up the min boundary on the last qualifier by one to ^(?=\D*\d)[a-z][a-z0-9]{3,7}$ the pattern fails again. Now if you change the zero to infinity qualifier in the lookahead to one to infinity qualifier in the modified pattern so you get ^(?=\D+\d)[a-z][a-z0-9]{3,7}$ the pattern matches again. Bump up the min boundary of this new pattern to ^(?=\D+\d)[a-z][a-z0-9]{4,7}$ Bugaboo! The pattern fails again.

Now I don't have access to the source code so this is just supposition but here's what I believe is happening. I think the data examined by the lookahead is being stored in a stack structure. But just looking at the patterns that are working and failing it looks like once the lookahead is satisfied values when a qualifier is encountered in the consuming portion of the pattern, the lookahead's match is reconsidered with the minimum boundary of the qualifier in the lookahead popped off the stack of the lookahead's match. Let's look at the first pattern that worked

^(?=\D*\d)[a-z][a-z0-9]{2,7}$

OK we'll start with after the lookahead is matched. Now the lookahead is supposed to be non-consuming so the pointer should still be at the beginning of the string.

^(?=\D*\d) matches “abc1” Now the rest of the pattern matches normally until we get to the qualifier in the consuming portion of the pattern we are looking for at least two alphanumeric characters. At that point the lookahead match is reconsidered, the lower bound being zero, nothing is remove for the stack but the current character pointer now points to the character after the lookahead's match value of “abc1”.The qualified part of the pattern [a-z0-9]{2,7}$ can be satisfied by “23” so we get a match

Now if we do the same thing with ^(?=\D*\d)[a-z][a-z0-9]{3,7}$ and apply the same logic the regex fails because the qualifier in the consuming portion is look for at least 3 characters and there aren't that many if it tries to satisfy that part of the pattern after “abc1” in the test string.

Now let's look at ^(?=\D+\d)[a-z][a-z0-9]{3,7}$ with the same logic. The only difference from the previous pattern is the lower boundary of the lookahead qualifier. It's now 1. So if we pop 1 character of the stack of the lookahead's match of “abc1” we get “abc” leaving us with “123” to be matched by the consuming qualifier, which is just enough.

Now take ^(?=\D+\d)[a-z][a-z0-9]{4,7}$ apply the same logic, now the pattern fails because the consuming qualifier is look for as least 4 characters after the stack popping of lookahead's match.

If you changed the lookahead qualifier to {2,} the pattern would match. You can continue upping the qualifiers by one in the consuming portion to make the pattern not match then non-consuming part of the pattern to get the match so the behavior seems pretty consistent with my theory which the consuming qualifier is pointing to the end of the lookahead match and moving back the number of character of the lookahead minimum boundary. It also seem to explain the effect of my original encounter with the bug. It may as well as why the workaround of testing the length first with another avoids the bug because that consuming qualifier it that case is usually just , though + seem to work too which doesn't quite fit but consuming qualifiers with minimum boundaries of zero or one don't seem to be effected in any case. In all of the above test cased those values were below the minimal threshold of every successful test. However the test cases above has only one lookahead that doesn't backtrack, who know what influence backtracking, additional lookaheads or how addition qualifiers in the consuming parts pattern would be effected. Now while the test values support my theory I can't say for sure that things are happening exactly the way I've laid out. The actual mechanics may be different but whatever is happening under the hood clearly pointers are being corrupted and the regex engine is loosing it's place

The thing that was so confusing about this was it only kicks in with a qualifier in the consuming portion is encountered but if there was something between the lookahead and the qualified portion it match normally. So this is something it's really hard to make test cases for because you get these ghost value popping up latter on in the test than I expected. Not to mention the pattern itself is correct so even with a tool that will let you step through the matching you don't get this behavior unless that tool is using VBScript's regex engine and I haven't seen such a tool for that engine.

If you are using lookaheads with JavaScript client-side then you are going to be susceptible to the bug in IE because it will use the Jscript engine. And while you should always validate server-side if VBSCript or Jscript is your server-side language you are still at risk. So a platform like classic ASP which uses both of those languages by default is at risk client and server side, but a platform like PHP while it still suffer the bug client-side for IE, should work correctly server-side which is using a different regex engine. Same goes for non-web clients using the JScript/VBScript DLL.

The workaround for the strong password type of regex is when using lookaheads to include the upper boundary test(s) before the no upper boundary test then use .* to consume characters. The bound test should keep the pattern from running forever. However depending on the complexity of your criteria this may not always be an option but try it first anyway.

This is a C# 2.0 enhancement of a C# port of YUI Compressor's  CSS minification code

I got a little carried away with ideas for this, they were all regex based which really is what motivated me to work on it. However after I thought I was done I learned not everything worked. It did what I wanted it to do but what I wanted wasn't the correct thing. I really should have just stopped with my original ideas.

The last idea for my original changes was to take 2 or more individual subset properties and write them in shorthand notation of the main property they were a subset of. Well I got that to working. But upon testing I learned something new about CSS that I didn't know. Basically that what I was doing could alter the behavior of the presentation. Which was disappointing because I put a lot of energy into getting the results I was after.

So it looked as all of that code was going to go to waste. But there was one scenario that what I was trying to do was alright. So the code wasn't completely wasted. The one scenario was if all the subset properties are declared then combining them is fine. I didn't bother changing the regexes I wrote for this but I cleaned up some of the code. Though it would have worked as is some of the things being checked were now unnecessary.

using System;
using System.Collections;
using System.Collections.Generic;
using System.Globalization;
using System.Text;
using System.Text.RegularExpressions;

namespace CSSMinify
{
    class CSSMinify
    {
        public static Hashtable shortColorNames = new Hashtable();
        public static Hashtable shortHexColors = new Hashtable();
        public static string Minify(string css)
        {
            return Minify(css, 0);
        }
        public static string Minify(string css, int columnWidth)
        {
            // BSD License http://developer.yahoo.net/yui/license.txt
            // New css tests and regexes by Michael Ash

            createHashTable();
            MatchEvaluator rgbDelegate = new MatchEvaluator(RGBMatchHandler);
            MatchEvaluator shortColorNameDelegate = new MatchEvaluator(ShortColorNameMatchHandler);
            MatchEvaluator shortColorHexDelegate = new MatchEvaluator(ShortColorHexMatchHandler);
            css = RemoveCommentBlocks(css);
            css = Regex.Replace(css, @"\s+", " "); //Normalize whitespace
            css = Regex.Replace(css, @"\x22\x5C\x22}\x5C\\x22\x22", "___PSEUDOCLASSBMH___"); //hide Box model hack
            /* Remove the spaces before the things that should not have spaces before them.
               But, be careful not to turn "p :link {...}" into "p:link{...}"
            */
            css = Regex.Replace(css, @"(?#no preceding space needed)\s+((?:[!{};>+()\],])|(?<={[^{}]*):(?=[^}]*}))", "$1");
            css = Regex.Replace(css, @"([!{}:;>+([,])\s+", "$1");  // Remove the spaces after the things that should not have spaces after them.
            css = Regex.Replace(css, @"([^;}])}", "$1;}");    // Add the semicolon where it's missing.
            css = Regex.Replace(css, @"(\d+)\.0+(p(?:[xct])|(?:[cem])m|%|in|ex)\b", "$1$2"); // Remove .0 from size units x.0em becomes xem
            css = Regex.Replace(css, @"([\s:])(0)(px|em|%|in|cm|mm|pc|pt|ex)\b", "$1$2"); // Remove unit from zero
            //New test
            //Font weights
            css = Regex.Replace(css, @"(?<=font-weight:)normal\b", "400");
            css = Regex.Replace(css, @"(?<=font-weight:)bold\b", "700");
            //Thought this was a good idea but properties of a set not defined get element defaults. This is reseting them. css = ShortHandProperty(css);
            css = ShortHandAllProperties(css);
            //css = Regex.Replace(css, @":(\s*0){2,4}\s*;", ":0;"); // if all parameters zero just use 1 parameter
            // if all 4 parameters the same unit make 1 parameter
            css = Regex.Replace(css, @"(?<!background-position\s*):\s*(inherit|auto|0|(?:(?:\d*\.?\d+(?:p(?:[xct])|(?:[cem])m|%|in|ex))))(\s+\1){1,3};", ":$1;", RegexOptions.IgnoreCase);
            // if has 4 parameters and top unit = bottom unit and right unit = left unit make 2 parameters
            css = Regex.Replace(css, @":\s*((inherit|auto|0|(?:(?:\d*\.?\d+(?:p(?:[xct])|(?:[cem])m|%|in|ex))))\s+(inherit|auto|0|(?:(?:\d?\.?\d(?:p(?:[xct])|(?:[cem])m|%|in|ex)))))\s+\2\s+\3;", ":$1;", RegexOptions.IgnoreCase);
            // if has 4 parameters and top unit != bottom unit and right unit = left unit make 3 parameters
            css = Regex.Replace(css, @":\s*((?:(?:inherit|auto|0|(?:(?:\d*\.?\d+(?:p(?:[xct])|(?:[cem])m|%|in|ex))))\s+)?(inherit|auto|0|(?:(?:\d?\.?\d(?:p(?:[xct])|(?:[cem])m|%|in|ex))))\s+(?:0|(?:(?:\d?\.?\d(?:p(?:[xct])|(?:[cem])m|%|in|ex)))))\s+\2;", ":$1;", RegexOptions.IgnoreCase);
            //// if has 3 parameters and top unit = bottom unit make 2 parameters
            //css = Regex.Replace(css, @":\s*((0|(?:(?:\d?\.?\d(?:p(?:[xct])|(?:[cem])m|%|in|ex))))\s+(?:0|(?:(?:\d?\.?\d(?:p(?:[xct])|(?:[cem])m|%|in|ex)))))\s+\2;", ":$1;", RegexOptions.IgnoreCase);
            css = Regex.Replace(css, "background-position:0;", "background-position:0 0;");
            css = Regex.Replace(css, @"(:|\s)0+\.(\d+)", "$1.$2");
            //  Outline-styles and Border-sytles parameter reduction
            css = Regex.Replace(css, @"(outline|border)-style\s*:\s*(none|hidden|d(?:otted|ashed|ouble)|solid|groove|ridge|inset|outset)(?:\s+\2){1,3};", "$1-style:$2;", RegexOptions.IgnoreCase);

            css = Regex.Replace(css, @"(outline|border)-style\s*:\s*((none|hidden|d(?:otted|ashed|ouble)|solid|groove|ridge|inset|outset)\s+(none|hidden|d(?:otted|ashed|ouble)|solid|groove|ridge|inset|outset ))(?:\s+\3)(?:\s+\4);", "$1-style:$2;", RegexOptions.IgnoreCase);

            css = Regex.Replace(css, @"(outline|border)-style\s*:\s*((?:(?:none|hidden|d(?:otted|ashed|ouble)|solid|groove|ridge|inset|outset)\s+)?(none|hidden|d(?:otted|ashed|ouble)|solid|groove|ridge|inset|outset )\s+(?:none|hidden|d(?:otted|ashed|ouble)|solid|groove|ridge|inset|outset ))(?:\s+\3);", "$1-style:$2;", RegexOptions.IgnoreCase);

            css = Regex.Replace(css, @"(outline|border)-style\s*:\s*((none|hidden|d(?:otted|ashed|ouble)|solid|groove|ridge|inset|outset)\s+(?:none|hidden|d(?:otted|ashed|ouble)|solid|groove|ridge|inset|outset ))(?:\s+\3);", "$1-style:$2;", RegexOptions.IgnoreCase);

            //  Outline-color and Border-color parameter reduction
            css = Regex.Replace(css, @"(outline|border)-color\s*:\s*((?:\#(?:[0-9A-F]{3}){1,2})|\S+)(?:\s+\2){1,3};", "$1-color:$2;", RegexOptions.IgnoreCase);

            css = Regex.Replace(css, @"(outline|border)-color\s*:\s*(((?:\#(?:[0-9A-F]{3}){1,2})|\S+)\s+((?:\#(?:[0-9A-F]{3}){1,2})|\S+))(?:\s+\3)(?:\s+\4);", "$1-color:$2;", RegexOptions.IgnoreCase);

            css = Regex.Replace(css, @"(outline|border)-color\s*:\s*((?:(?:(?:\#(?:[0-9A-F]{3}){1,2})|\S+)\s+)?((?:\#(?:[0-9A-F]{3}){1,2})|\S+)\s+(?:(?:\#(?:[0-9A-F]{3}){1,2})|\S+))(?:\s+\3);", "$1-color:$2;", RegexOptions.IgnoreCase);

            // Shorten colors from rgb(51,102,153) to #336699
            // This makes it more likely that it'll get further compressed in the next step.
            css = Regex.Replace(css, @"rgb\s*\x28((?:25[0-5])|(?:2[0-4]\d)|(?:[01]?\d?\d))\s*,\s*((?:25[0-5])|(?:2[0-4]\d)|(?:[01]?\d?\d))\s*,\s*((?:25[0-5])|(?:2[0-4]\d)|(?:[01]?\d?\d))\s*\x29", rgbDelegate);
            css = Regex.Replace(css, @"(?<![\x22\x27=]\s*)\#(?:([0-9A-F])\1)(?:([0-9A-F])\2)(?:([0-9A-F])\3)", "#$1$2$3", RegexOptions.IgnoreCase);
            // Replace hex color code with named value is shorter
            css = Regex.Replace(css, @"(?<=color\s*:\s*.*)\#(?<hex>f00)\b", "red", RegexOptions.IgnoreCase);
            css = Regex.Replace(css, @"(?<=color\s*:\s*.*)\#(?<hex>[0-9a-f]{6})", shortColorNameDelegate, RegexOptions.IgnoreCase);
            css = Regex.Replace(css, @"(?<=color\s*:\s*)\b(Black|Fuchsia|LightSlateGr[ae]y|Magenta|White|Yellow)\b", shortColorHexDelegate, RegexOptions.IgnoreCase);

            // Remove empty rules.
            css = Regex.Replace(css, @"[^}]+{;}", "");
            //Remove semicolon of last property
            css = Regex.Replace(css, ";(})", "$1");
            if (columnWidth > 0)
            {
                css = BreakLines(css, columnWidth);
            }
            return css;
        }
        private static string RemoveCommentBlocks(string input)
        {
            int startIndex = 0;
            int endIndex = 0;
            bool iemac = false;
            startIndex = input.IndexOf(@"/*", startIndex);
            while (startIndex >= 0)
            {
                endIndex = input.IndexOf(@"*/", startIndex + 2);
                if (endIndex >= startIndex + 2)
                {
                    if (input[endIndex - 1] == '\\')
                    {
                        startIndex = endIndex + 2;
                        iemac = true;
                    }
                    else if (iemac)
                    {
                        startIndex = endIndex + 2;
                        iemac = false;
                    }
                    else
                    {
                        input = input.Remove(startIndex, endIndex + 2 - startIndex);
                    }
                }
                startIndex = input.IndexOf(@"/*", startIndex);
            }
            return input;
        }
        private static String RGBMatchHandler(Match m)
        {
            int val = 0;
            StringBuilder hexcolor = new StringBuilder("#");
            for (int index = 1; index <= 3; index += 1)
            {
                val = Int32.Parse(m.Groups[index].Value);
                hexcolor.Append(val.ToString("x2"));
            }
            return hexcolor.ToString();
        }
        private static string BreakLines(string css, int columnWidth)
        {
            int i = 0;
            int start = 0;
            StringBuilder sb = new StringBuilder(css);
            while (i < sb.Length)
            {
                char c = sb[i++];
                if (c == '}' && i - start > columnWidth)
                {
                    sb.Insert(i, '\n');
                    start = i;
                }
            }
            return sb.ToString();

        }
        private static string ReplaceNonEmpty(string inputText, string replacementText)
        {
            if (replacementText.Trim() != string.Empty)
            {
                inputText = string.Format(" {0}", replacementText);
            }
            return inputText;
        }
        private static string ShortColorNameMatchHandler(Match m)
        {
            // This function replace hex color values named colors if the name is shorter than the hex code
            string returnValue = m.Value;
            if (shortColorNames.ContainsKey(m.Groups["hex"].Value))
            {
                returnValue = shortColorNames[m.Groups["hex"].Value].ToString();
            }
            return returnValue;
        }
        private static string ShortColorHexMatchHandler(Match m)
        {
            //This function replaces named values with there shorter hex equivalent
            return shortHexColors[m.Value.ToString().ToLower()].ToString();
        }
        private static void createHashTable()
        {
            //Color names shorter than hex notation. Except for red.
            shortColorNames.Add("F0FFFF".ToLower(), "Azure".ToLower());
            shortColorNames.Add("F5F5DC".ToLower(), "Beige".ToLower());
            shortColorNames.Add("FFE4C4".ToLower(), "Bisque".ToLower());
            shortColorNames.Add("A52A2A".ToLower(), "Brown".ToLower());
            shortColorNames.Add("FF7F50".ToLower(), "Coral".ToLower());
            shortColorNames.Add("FFD700".ToLower(), "Gold".ToLower());
            shortColorNames.Add("808080".ToLower(), "Grey".ToLower());
            shortColorNames.Add("008000".ToLower(), "Green".ToLower());
            shortColorNames.Add("4B0082".ToLower(), "Indigo".ToLower());
            shortColorNames.Add("FFFFF0".ToLower(), "Ivory".ToLower());
            shortColorNames.Add("F0E68C".ToLower(), "Khaki".ToLower());
            shortColorNames.Add("FAF0E6".ToLower(), "Linen".ToLower());
            shortColorNames.Add("800000".ToLower(), "Maroon".ToLower());
            shortColorNames.Add("000080".ToLower(), "Navy".ToLower());
            shortColorNames.Add("808000".ToLower(), "Olive".ToLower());
            shortColorNames.Add("FFA500".ToLower(), "Orange".ToLower());
            shortColorNames.Add("DA70D6".ToLower(), "Orchid".ToLower());
            shortColorNames.Add("CD853F".ToLower(), "Peru".ToLower());
            shortColorNames.Add("FFC0CB".ToLower(), "Pink".ToLower());
            shortColorNames.Add("DDA0DD".ToLower(), "Plum".ToLower());
            shortColorNames.Add("800080".ToLower(), "Purple".ToLower());
            shortColorNames.Add("FA8072".ToLower(), "Salmon".ToLower());
            shortColorNames.Add("A0522D".ToLower(), "Sienna".ToLower());
            shortColorNames.Add("C0C0C0".ToLower(), "Silver".ToLower());
            shortColorNames.Add("FFFAFA".ToLower(), "Snow".ToLower());
            shortColorNames.Add("D2B48C".ToLower(), "Tan".ToLower());
            shortColorNames.Add("008080".ToLower(), "Teal".ToLower());
            shortColorNames.Add("FF6347".ToLower(), "Tomato".ToLower());
            shortColorNames.Add("EE82EE".ToLower(), "Violet".ToLower());
            shortColorNames.Add("F5DEB3".ToLower(), "Wheat".ToLower());

            // Hex notation shorter than named value
            shortHexColors.Add("black", "#000");
            shortHexColors.Add("fuchsia", "#f0f");
            shortHexColors.Add("lightSlategray", "#789");
            shortHexColors.Add("lightSlategrey", "#789");
            shortHexColors.Add("magenta", "#f0f");
            shortHexColors.Add("white", "#fff");
            shortHexColors.Add("yellow", "#ff0");
        }
        private static string ShortHandAllProperties(string css)
        {
            /*
             * This function searchs for properties specifying all the individual properties of a property type
             * and reduces it to a single property use shorthand notation
             */
            Regex reCSSBlock = new Regex("{[^{}]*}");
            Regex reTRBL1 = new Regex(@"(?<fullProperty>(?:(?<property>padding)-(?<position>top|right|bottom|left)))\s*:\s*(?<unit>[\w.]+);?", RegexOptions.IgnoreCase);
            Regex reTRBL2 = new Regex(@"(?<fullProperty>(?:(?<property>margin)-(?<position>top|right|bottom|left)))\s*:\s*(?<unit>[\w.]+);?", RegexOptions.IgnoreCase);
            Regex reTRBL3 = new Regex(@"(?<fullProperty>(?<property>border)-(?<position>top|right|bottom|left)(?<property2>-(?:color)))\s*:\s*(?<unit>[#\w.]+);?", RegexOptions.IgnoreCase);
            Regex reTRBL4 = new Regex(@"(?<fullProperty>(?<property>border)-(?<position>top|right|bottom|left)(?<property2>-(?:style)))\s*:\s*(?<unit>none|hidden|d(?:otted|ashed|ouble)|solid|groove|ridge|inset|outset);?", RegexOptions.IgnoreCase);
            Regex reTRBL5 = new Regex(@"(?<fullProperty>(?<property>border)-(?<position>top|right|bottom|left)(?<property2>-(?:width)))\s*:\s*(?<unit>[\w.]+);?", RegexOptions.IgnoreCase);
            Regex reListStyle = new Regex(@'list-style-(?<style>type|image|position)\s*:\s*(?<unit>[^};]+);?', RegexOptions.IgnoreCase);
            Regex reFont = new Regex(@"font-(?:(?:(?<fontProperty>family\b)\s*:\s*(?<fontPropertyValue>(?:\b[a-zA-Z]+(-[a-zA-Z]+)?\b|\x22[^\x22]+\x22)(?:\s*,\s*(?:\b[a-zA-Z]+(-[a-zA-Z]+)?\b|\x22[^\x22]+\x22))*)\b)|
(?:(?<fontProperty>style\b)\s*:\s*(?<fontPropertyValue>normal|italic|oblique|inherit))|
(?:(?<fontProperty>variant\b)\s*:\s*(?<fontPropertyValue>normal|small-caps|inherit))|
(?:(?<fontProperty>weight\b)\s*:\s*(?<fontPropertyValue>normal|bold|(?:bold|light)er|[1-9]00|inherit))|
(?:(?<fontProperty>size\b)\s*:\s*(?<fontPropertyValue>(?:(?:xx?-)?(?:small|large))|medium|(?:\d*\.?\d+(?:%|(p(?:[xct])|(?:[cem])m|in|ex))\b)|inherit|\b0\b)))\s*;?", (RegexOptions.IgnoreCase | RegexOptions.IgnorePatternWhitespace));
            Regex reBackGround = new Regex(@"background-(?:
(?:(?<property>color)\s*:\s*(?<unit>transparent|inherit|(?:(?:\#(?:[0-9A-F]{3}){1,2})|\S+)))|
(?:(?<property>image)\s*:\s*(?<unit>none|inherit|(?:url\s*\([^()]+\))))|
(?:(?<property>repeat)\s*:\s*(?<unit>no-repeat|inherit|repeat(?:-[xy])))|
(?:(?<property>attachment)\s*:\s*(?<unit>scroll|inherit|fixed))|
(?:(?<property>position)\s*:\s*(?<unit>((?<horizontal>left | center | right|(?:0|(?:(?:\d*\.?\d+(?:p(?:[xct])|(?:[cem])m|%|in|ex)))))\s+(?<vertical>top | center | bottom |(?:0|(?:(?:\d*\.?\d+(?:p(?:[xct])|(?:[cem])m|%|in|ex))))))|
    ((?<vertical>top | center | bottom )\s+(?<horizontal>left | center | right ))|
    ((?<horizontal>left | center | right )|(?<vertical>top | center | bottom ))))
);?", (RegexOptions.IgnoreCase | RegexOptions.IgnorePatternWhitespace | RegexOptions.ExplicitCapture));
            MatchCollection mcBlocks = reCSSBlock.Matches(css);
            foreach (Match mBlock in mcBlocks)
            {
                string strBlock = mBlock.Value;
                HasAllPositions(reTRBL1, ref strBlock);
                HasAllPositions(reTRBL2, ref strBlock);
                HasAllPositions(reTRBL3, ref strBlock);
                HasAllPositions(reTRBL4, ref strBlock);
                HasAllPositions(reTRBL5, ref strBlock);
                HasAllListStyle(reListStyle, ref strBlock);
                HasAllFontProperties(reFont, ref strBlock);
                HasAllBackGroundProperties(reBackGround, ref strBlock);
                css = css.Replace(mBlock.Value, strBlock);
            }
            return css;
        }
        private static void HasAllBackGroundProperties(Regex re, ref string CSSText)
        {
            {
                MatchCollection mcProperySet = re.Matches(CSSText);
                int z = 5;
                if (mcProperySet.Count == z)
                {

                    int y = 0;
                    for (int x = 0; x < z; x = x + 1)
                    {
                        switch (mcProperySet[x].Groups["property"].Value)
                        {
                            case "color":
                                y = y + 1;
                                break;
                            case "image":
                                y = y + 2;
                                break;
                            case "repeat":
                                y = y + 4;
                                break;
                            case "attachment":
                                y = y + 8;
                                break;
                            case "position":
                                y = y + 16;
                                break;
                        }
                    }
                    if (y == 31)
                    {
                        CSSText = ShortHandBackGroundReplaceV2(mcProperySet, re, CSSText);
                    }
                }
            }
        }
        private static void HasAllFontProperties(Regex re, ref string CSSText)
        {
            {
                MatchCollection mcProperySet = re.Matches(CSSText);
                int z = 5;
                if (mcProperySet.Count == z)
                {

                    int y = 0;
                    for (int x = 0; x < z; x = x + 1)
                    {
                        switch (mcProperySet[x].Groups["fontProperty"].Value)
                        {
                            case "style":
                                y = y + 1;
                                break;
                            case "variant":
                                y = y + 2;
                                break;
                            case "weight":
                                y = y + 4;
                                break;
                            case "size":
                                y = y + 8;
                                break;
                            case "family":
                                y = y + 16;
                                break;
                        }
                    }
                    if (y == 31)
                    {
                        CSSText = ShortHandFontReplaceV2(mcProperySet, re, CSSText);
                    }
                }
            }
        }
        private static void HasAllListStyle(Regex re, ref string CSSText)
        {
            {
                int z = 3;
                MatchCollection mcProperySet = re.Matches(CSSText);
                if (mcProperySet.Count == z)
                {

                    int y = 0;
                    for (int x = 0; x < z; x = x + 1)
                    {
                        switch (mcProperySet[x].Groups["style"].Value)
                        {
                            case "type":
                                y = y + 1;
                                break;
                            case "image":
                                y = y + 2;
                                break;
                            case "position":
                                y = y + 4;
                                break;

                        }
                    }
                    if (y == 7)
                    {
                        CSSText = ShortHandListReplaceV2(mcProperySet, re, CSSText);
                    }
                }
            }
        }
        private static void HasAllPositions(Regex re, ref string CSSText)
        {
            {
                MatchCollection mcProperySet = re.Matches(CSSText);
                if (mcProperySet.Count == 4)
                {

                    int y = 0;
                    for (int x = 0; x < 4; x = x + 1)
                    {
                        switch (mcProperySet[x].Groups["position"].Value)
                        {
                            case "top":
                                y = y + 1;
                                break;
                            case "right":
                                y = y + 2;
                                break;
                            case "bottom":
                                y = y + 4;
                                break;
                            case "left":
                                y = y + 8;
                                break;
                        }
                    }
                    if (y == 15)
                    {
                        CSSText = ShortHandReplaceV2(mcProperySet, re, CSSText);
                    }
                }
            }
        }
        private static string ShortHandFontReplaceV2(MatchCollection mcProperySet, Regex re, string InputText)
        {
            /*
             * This Function replaces the individual font properties with a single entry
             * */
            string strFamily, strStyle, strVariant, strWeight, strSize;
            Regex reLineHeight = new Regex(@"line-height\s*:\s*((?:\d*\.?\d+(?:%|(p(?:[xct])|(?:[cem])m|in|ex)\b)?)|normal|inherit);?", RegexOptions.IgnoreCase);
            strFamily = string.Empty;
            strStyle = string.Empty;
            strVariant = string.Empty;
            strWeight = string.Empty;
            strSize = string.Empty;
            string strStyle_Variant_Weight = string.Empty;
            foreach (Match mProperty in mcProperySet)
            {
                switch (mProperty.Groups[""].Value)
                {
                    case "family":
                        strFamily = string.Format(" {0}", mProperty.Groups["fontPropertyValue"].Value);
                        break;
                    case "size":
                        if (reLineHeight.IsMatch(InputText))
                        {
                            Match m = reLineHeight.Match(InputText);
                            if (m.Groups[1].Value != "normal")
                            {
                                strSize = String.Format("/{0}", m.Groups[1].Value);
                            }
                            InputText = reLineHeight.Replace(InputText, string.Empty);
                        }
                        strSize = string.Format(" {0}{1}", mProperty.Groups["fontPropertyValue"].Value, strSize);
                        if (strSize == "medium")
                        {
                            strSize = string.Empty;
                        }
                        break;
                    case "style":
                    case "variant":
                    case "weight":
                        if (mProperty.Groups["fontPropertyValue"].Value != "normal")
                        {
                            strStyle_Variant_Weight += string.Format(" {0}", mProperty.Groups["fontPropertyValue"].Value);
                        } break;

                }
            }

            string strShortcut;
            string strProperties = string.Format("{0}{1}{2};", strStyle_Variant_Weight, strVariant, strWeight, strSize, strFamily);
            strShortcut = string.Format("font:{0}", strProperties.Trim());
            string strNewBlock = re.Replace(InputText, "");
            strNewBlock = strNewBlock.Insert(1, strShortcut);
            return strNewBlock;
        }
        private static string ShortHandBackGroundReplaceV2(MatchCollection mcProperySet, Regex re, string InputText)
        {
            /*
             * This Function replaces the individual background properties with a single entry
             * */
            string strColor, strImage, strRepeat, strAttachment, strPosition;
            strColor = string.Empty;
            strImage = string.Empty;
            strRepeat = string.Empty;
            strAttachment = string.Empty;
            strPosition = string.Empty;
            foreach (Match mProperty in mcProperySet)
            {
                switch (mProperty.Groups["property"].Value)
                {
                    case "color":
                        if (mProperty.Groups["unit"].Value != "transparent")
                        {
                            strColor = string.Format(" {0}", mProperty.Groups["unit"].Value);
                        }
                        break;
                    case "image":
                        if (mProperty.Groups["unit"].Value != "none")
                        {
                            strImage = string.Format(" {0}", mProperty.Groups["unit"].Value);
                        }
                        break;
                    case "repeat":
                        if (mProperty.Groups["unit"].Value != "repeat")
                        {
                            strRepeat = string.Format(" {0}", mProperty.Groups["unit"].Value);
                        } break;
                    case "attachment":
                        if (mProperty.Groups["unit"].Value != "scroll")
                        {
                            strAttachment = string.Format(" {0}", mProperty.Groups["unit"].Value);
                        }
                        break;
                    case "position":
                        if (mProperty.Groups["unit"].Value != "0% 0%")
                        {
                            strPosition = string.Format(" {0}", mProperty.Groups["unit"].Value);
                        }
                        break;
                }
            }

            string strShortcut;
            string strProperties = string.Format("{0}{1}{2}{3}{4};", strColor, strImage, strRepeat, strAttachment, strPosition);
            strShortcut = string.Format("background:{0}", strProperties.Trim());
            string strNewBlock = re.Replace(InputText, "");
            strNewBlock = strNewBlock.Insert(1, strShortcut);
            return strNewBlock;
        }
        private static string ShortHandReplaceV2(MatchCollection mcProperySet, Regex reTRBL1, string InputText)
        {
            // Replace method for regexes used in ShortHand property method for properties with top, right, bottom and left sub properties.
            string strTop, strRight, strBottom, strLeft;
            strTop = string.Empty;
            strRight = string.Empty;
            strBottom = string.Empty;
            strLeft = string.Empty;
            string strProperty;
            strProperty = string.Format("{0}{1}", mcProperySet[0].Groups["property"].Value, mcProperySet[0].Groups["property2"].Value);
            foreach (Match mProperty in mcProperySet)
            {
                switch (mProperty.Groups["position"].Value)
                {
                    case "top":
                        strTop = mProperty.Groups["unit"].Value;
                        break;
                    case "right":
                        strRight = mProperty.Groups["unit"].Value;
                        break;
                    case "bottom":
                        strBottom = mProperty.Groups["unit"].Value;
                        break;
                    case "left":
                        strLeft = mProperty.Groups["unit"].Value;
                        break;
                }

            }

            string strShortcut = string.Format("{0}:{1} {2} {3} {4};", strProperty, strTop, strRight, strBottom, strLeft);
            string strNewBlock = reTRBL1.Replace(InputText, "");
            strNewBlock = strNewBlock.Insert(1, strShortcut);
            return strNewBlock;
        }
        private static string ShortHandListReplaceV2(MatchCollection mcProperySet, Regex re, string InputText)
        {
            /*
             * This Function replaces the individual list properties with a single entry
             * */
            string strType, strPosition, strImage;
            strType = string.Empty;
            strPosition = string.Empty;
            strImage = string.Empty;
            foreach (Match mProperty in mcProperySet)
            {
                switch (mProperty.Groups["style"].Value)
                {
                    case "type":
                        if (mProperty.Groups["unit"].Value != "disc")
                        {
                            strType = mProperty.Groups["unit"].Value;
                        }
                        break;
                    case "position":
                        if (mProperty.Groups["unit"].Value != "outside")
                        {
                            strPosition = string.Format(" {0}", mProperty.Groups["unit"].Value);
                        }
                        break;
                    case "style":
                        if (mProperty.Groups["unit"].Value != "none")
                        {
                            strImage = string.Format(" {0}", mProperty.Groups["unit"].Value);
                        }
                        break;
                }

            }

            string strShortcut = string.Format("list-style:{0}{1}{2};", strType, strPosition, strImage);
            string strNewBlock = re.Replace(InputText, "");
            strNewBlock = strNewBlock.Insert(1, strShortcut);
            return strNewBlock;
        }
    }
}
 

NOTE: All the regex referenced on this page written by me are using IgnoreCase = true

I was looking at the regexes used in the YUI Compressor to minify CSS and came up with a couple of more that I think could help the process. The code and port I was looking at was already trimming unneeded zeros used for the top, right, bottom, left values with a simple string replace. But there were three separate replaces being done. It was pretty simple to come up with a regex to handle all the cases

(Pseudo code)

string.Replace(":0 0 0 0;","0;")
string.Replace(":0 0 0;","0;")
string.Replace(":0 0;","0;")

becomes

Regex.Replace(input,":(\s*0)(\s+0){0,3}\s*;",":0;")

Pretty simple but of course I thought why stop there so I came up with a regex for all numbers

:\s*(0|(?:(?:\d*\.?\d+(?:p(?:[xct])|(?:[cem])m|%|in|ex))))(\s+\1){1,3};

The replacement string is simply ":$1;"

Once this was done next on the list was to handle cases when all the numbers are not the all the same.

For those who don't know CSS this part of the syntax basically says of the 4 possible values

1) if only one value is specified the other 3 are implied to the same value (X = X X X X)
2) if two values are specified the 1st implies the third the second implies the 4th (X Y = X Y X Y)
3) if three values are specified the second implies the 4th (X Y Z = X Y Z Y)
 
Of course minifying you want to use the shorter syntax so the following regexes convert the longer to the shorter.
The replacement string for all is the same as above "$1;"

# 4 parameters to 2 x y x y to x y

:\s*((0|(?:(?:\d?\.?\d(?:p(?:[xct])|(?:[cem])m|%|in|ex))))\s+(0|(?:(?:\d?\.?\d(?:p(?:[xct])|(?:[cem])m|%|in|ex)))))\s+\2\s+\3;

# 4 to 3 (x y z y to x y z) or 3 to 2 (x y x to x y)

:\s*((?:(?:0|(?:(?:\d?\.?\d(?:p(?:[xct])|(?:[cem])m|%|in|ex))))\s+)?(0|(?:(?:\d?\.?\d(?:p(?:[xct])|(?:[cem])m|%|in|ex))))\s+(?:0|(?:(?:\d?\.?\d(?:p(?:[xct])|(?:[cem])m|%|in|ex)))))\s+\2;

Along those same lines I came up with similar patterns for border-style, outline-style, border-color and outline-color

border-style/outline-style

The replacement string for all is “$1-style:$2;”

(outline|border)-style\s*:\s*(none|hidden|d(?:otted|ashed|ouble)|solid|groove|ridge|inset|outset )(?:\s+\2){1,3};

(outline|border)-style\s*:\s*((none|hidden|d(?:otted|ashed|ouble)|solid|groove|ridge|inset|outset )\s+(none|hidden|d(?:otted|ashed|ouble)|solid|groove|ridge|inset|outset ))(?:\s+\3)(?:\s+\4);

(outline|border)-style\s*:\s*((?:(?:none|hidden|d(?:otted|ashed|ouble)|solid|groove|ridge|inset|outset )\s+)?(none|hidden|d(?:otted|ashed|ouble)|solid|groove|ridge|inset|outset )\s+(?:none|hidden|d(?:otted|ashed|ouble)|solid|groove|ridge|inset|outset ))(?:\s+\3);

border-color/outline-color

The replacement string for all is “$1-color:$2;”

(outline|border)-color\s*:\s*((?:\#(?:[0-9A-F]{3}){1,2})|\S+)(?:\s+\2){1,3};

(outline|border)-color\s*:\s*(((?:\#(?:[0-9A-F]{3}){1,2})|\S+)\s+((?:\#(?:[0-9A-F]{3}){1,2})|\S+))(?:\s+\3)(?:\s+\4);

(outline|border)-color\s*:\s*((?:(?:(?:\#(?:[0-9A-F]{3}){1,2})|\S+)\s+)?((?:\#(?:[0-9A-F]{3}){1,2})|\S+)\s+(?:(?:\#(?:[0-9A-F]{3}){1,2})|\S+))(?:\s+\3);

I also came up with a couple of more regexes to replace some code, but as couple of these use look-behinds they are not as portable.

These work in .Net which supports variable length lookbehinds

This pattern

\s+((?:[!{};>+()\],])|(?<={[^{}]*):(?=[^}]*}))

Is used to find character preceeded by an unneed whitespace taking care not to find colon's that are used as psuedo-selectors or psuedo-classes. Replacement string would be $1

This pattern

(?<![\x22\x27=]\s*)\#([0-9A-F])\1([0-9A-F])\2([0-9A-F])\3

is used for reducing hexadecimal values from AABBCC to ABC, the replacement string would be $1$2$3.

The patten I created was to find RGB values of th format rgb(x, y, z) where x, y and z are the integers in the range of 0-255.

rgb\s*\x28((?:25[0-5])|(?:2[0-4]\d)|(?:[01]?\d?\d))\s*,\s*((?:25[0-5])|(?:2[0-4]\d)|(?:[01]?\d?\d))\s*,\s*((?:25[0-5])|(?:2[0-4]\d)|(?:[01]?\d?\d))\s*\x29

The pattern I used was just stricter than the existing one and put each value as a group so I could just work with them with further processing. The match itself is passed to a matchevaluator to do the decimal to hex conversion.

I did a little alpha testing but most of the CSS I've written doesn't have the stuff I'm testing for. So beta testing is definitely in order. Also I don't really know CSS hacks so I don't know if any of this will have adverse effect on hacks. Other than the top/left/bottom/right parameters I mostly trying to duplicate the existing effects. I'm going to pass this info on the YUI Compressor maintainer to see if they can make use of any of this.

I see this all the time on forums where posters asking for help almost never post any sample data, and most of the few that do post a sample they made up on the spot. Over on they RegexAdvice construction forum we've added guidelines for posting a question. The whole point of the guidelines are to expedite getting questions answered. One of the things we ask is that posters from a sample of the actual data they are trying to match. We explicitly ask they don't make up a sample. Unfortunately most people don't follow this request, along with others, in the guidelines, which generally result in those questions taking longer to get answered. On this particular point not supplying any sample is worst offense because anyone trying to help you can't see what you are working with and can only give an untested guess solution It like calling a mechanic on the phone and saying “My car isn't running right” leaving your number and waiting for him to call back to tell you how to fix it. He can call back and tell you to change your oil, but without more detail he can't know that will fix your problem. However making up a sample doesn't help much either and is almost just as bad. I know what people think they are being helpful and not boring people with the bloody details by providing a simple example to work with but generally it's not helpful since it doesn't represent their real problem. In regex construction details matter. Often the made up sample is too simple. So what they end of getting is a solution that works great for their sample but doesn't handle their real problem well, if at all. So now you are back where you started. This is like calling your mechanic on the phone and saying “My car making a funny noise.” While it's less vague than “isn't running right” it's still vague. You may know exactly what kind of noise, and how loud it is and where it coming from but that sentence doesn't communicate any of that. Made up samples are about the same when asking for regex help.

The majority of regular expression problems are content sensitive. Meaning the approach of how to match the target string may depend on where it is the the source string. Now sometime the target string is unique from the surrounding text that the context seems irrelevant. Say you are trying to find a string of digits somewhere in your source data. And there just happens to be only one occurrence of digits are in your source. Well it that case a generic regex that find digits will be good enough.

Source : “abc345def” or “a125bcd”

Regex: \d+

Now I said the content seems irrelevant in this case but it is as relevant as ever. The context is digits only occur once in the string.

But relevance becomes clearer if you have a source file where there are two or more occurrences of text that mimic your target data but only one is actually the data you want. For example let say you want to find area codes of a phone number. Well if you agree that an area code is a 3 digit number the regex \d{3} might be all need. It depends. On what you ask? Come on you know, say it with me....”Your Source data!”

If your source data is like the case I mentioned first, where the only digits in the files are area codes or if there are other digit in the file that are not area codes those other numbers are less than three consecutive digits then the simple regex should work

Sample

1) AL 205, 256, 334

2) AK 907

3) AR 479

4) etc.

But if your source is a bunch phone numbers

1) Mom (555) 123-4567

2) Sis 1-555-234-4567

3) Friend 5553456789

Well now you have a problem. While you general task is still the same, you have to approach it differently because of the source. You can't simply look for 3 consecutive digits since that would return 3 matches for each line. But if we know how the data is structured we can focus on how the data we want sticks out from the rest. In the above sample text area code meets on of three conditions

1. 3 cons digits inside of parenthesis

2. first group of 3 consecutive digits in a string of 1-xxx-xxx-xxxx format

3. first 3 consecutive digits of a ten consecutive digit string

Now you can still match what you need in one regex but how you choose to approach it changes. Some regex authors will use look-around to examine the surrounding text, others will match the surrounding and use groups to pull out the desired data. Either way you have to expand your field of vision beyond what you are trying to match to address the problem.

But even in this simple example, understand how knowing what the source data look like is still important. Imagine someone asked you for help and you are trying to solve this problem without seeing the actual source above. If someone post (or ask) the question “I need a regex to match a US area code”

With No Sample: Any helpful regex author has to guess how your data is formatted. Some may assume it's a string of “(xxx) yyy-yyyy”, others may assume it's “xxx-yyy-yyyy”, some may even think it “State: xxx, xxx, xxx” and all might not it even consider “1-xxx-yyy-yyyy” or “xxxyyyyyyy”.

With Made up sample: “I have a string (555) 555-5555 and I need a regex to match a US area code”

Now regex author may assume all your phone numbers are in that nice format and that there are no variations. You'll most likely get a regex that match that format perfectly, but fail the other formats you didn't think were important enough to mention.

With Real Data: Regex authors don't have to guess they can look at your data analyze it themselves and even test edge cases and address issues you may have overlooked.

I hope this simple example shows how the same problem can have different approaches all because of the source. Where data in the wild is going to be much more complex than this which will only magnify the issue.

Now sometimes you can't provide real data because either its too much or private. With large chunk of data simply clip a portion containing what you are wanting to match. With private data making up sample is almost excusable but don't make up simple ones. Garble the real data but preserve the format, change names to literary or TV characters, change addresses to the address of some public office (City hall, post office) or landmark which this is essentially a made up sample but if you substitute values your samples won't be as bland and closer to your real data and edge cases.

Understanding your source data is crucial to your regex construction whether you are writing it yourself or asking for help.

Hello boys and girls. Wow it's been a while since I've done this. I want to touch on a very useful but often overlooked feature of regex, grouping. While I haven't been blogging I have been active on a message board here or there. A question I see quite often is “I want to find a match in a string but I don't want part of the match” or “I need the value of this portion of the string” Now often I see solutions to these type of questions that involve look-arounds. Any why they certainly work they aren't the only way to achieve the desired results. New regex users seem to believe they can only access the full match. Most regex engine support groups, where in a match you can access a certain portion of the full match. Groups are identified by parenthesis. Every pair of plain parenthesis is a group. For example the regex pattern

/(Hello)\x20(world)/

There are two groups in the regex. The regex itself matches the string “Hello world”, group 1 contains the string “Hello”, group 2 contains the string “world”. Note neither group contains the space between the two words but it is part of the full match. Most implementations of regular expressions allow you a ways to access these groups. They are contained in a collection inside the match object. Now you'll need to consult your regex documentation to know the exact layout but in most of these implementation there is a zero based collection where Item 0 is the full match and Item 1..n is whichever group element your regex contains, if any.

Now if you notice I said every pair of plain parenthesis is a group. The reason I stress plain is because there are other group constructs, the aforementioned look-arounds being some. Now support for the other constructs vary in implementations so again consult your regex documentation to see which ones you have. The other grouping constructs consist of a open parenthesis immediately followed by a question mark, which then is followed by the characters that define that particular grouping construct. Again consult your documentation to see which characters define what. I'm not going to go into all of them here but they basically fall into two categories. Capturing and non-Capturing. The plain parenthesis I've mentioned above are a capturing group. However capturing requires extra resources in some case you need the extra speed, but you still to group a certain part of the pattern together for either necessity or readability or both. This is where you'll what to using a non-capturing group (?:pattern), a open parenthesis followed immediately by a question mark followed immediately by a colon. The difference here is that the data matched in the group is not add to the collection of submatch in the Match object.

Taking our previous example and making the first group non-capturing

/(?:Hello)\x20(world)/

Where before we had two groups here we only have one. Group 1 contains the string”world”. Now this example is not a very practical use of a non-capturing group. Typically you'd use them in more complex regexes that have a grouping but you really don't care about the sub-matches.

One more quick thing about capturing groups basically each left parenthesis is the index of the sub-match in the groups collection. So if you have nested parenthesis count every (plain) left parenthesis to know which index to use to reference it. Some of the advance grouping constructs and regex options can affect the ordering but if you are using them hopefully you've read their effects so I won't go over that here.

/((Hello)|(Goodbye Cruel))\x20(world)/

The above regex has 4 capturing groups (not counting group 0). Can you find them? Now it should match either the string “Hello world” or “Goodbye Cruel world” Now I want to point out that not all the groups will participate in the match, but the are still part of the Groups collection. There will always be 4 groups, just one will always be empty. Which one depends on which string was matched.

If “Hello world” was matched the groups are

1. Hello

2. Hello

3. (empty)

4. world

If “Goodbye Cruel world” was matched the groups are

1. Goodbye Cruel

2. (empty)

3. Goodbye Cruel

4. world

in both case group 0 would be the full match

If you note in both case two groups contain the same value. Even if you need to know whether “Hello” or “Goodbye Cruel” was match, you certainly don't need to know it twice. Plus the inner parenthesis have different index you'd have to check if you want to use those. This is where you'd use the non-capturing group to simplify your groups collection.

/((?:Hello)|(?:Goodbye Cruel))\x20(world)/

Now we are back down to two groups. Group 1 contains either “Hello” or “Goodbye Cruel” depending on which string was matched. Group 2 always contains “world”

However keep in mind in some cases you'll want to use the inner index do determine which group was matched. So using non-capturing groups isn't necessarily a better thing it just depends on if you need to access those groups or not. But if you are not doing anything with them don't capture them.

These are just two of the basic grouping constructs and they are general supported across implementations of regex, but not always. But if they are you can use the to easily dissect larger matches.

There are times when regular expression you’ve written or someone written for you needs a little tweaking before you add it to your code and the tweaking is required because the syntax of the language conflicts with your regex.   For example when part of your regex pattern contains a double quote and the language you are using uses double quotes as string delimiters.  If you just cut and paste the pattern in your code the pattern’s quotation will terminate your string prematurely.  Now the code way to fix it is to escape the quotation in pattern. This solution requires altering the regex and how the character is escapes depends on the language being used.   The regex itself allows you to escape character with the \ character.  The language being used may or may not recognize that as escape character for its syntax.   And it may be confusing later when you look at the regex and can’t remember why you escaped a character that the pattern itself doesn’t need it, But there is another way. Hex values

Most regex implementations support a hex syntax \x##,  where # is a hex digit.

So if you use \x22 instead of double quote and \x27 for single quotes the regexes become more cookie cutter ready.

Another useful hex value is \x20 which is a space.   This is especially useful in .Net where there is an option on a regex to ignorewhitespace in the pattern.  Turning this option on allows end of line comments in the regex but with the exception of inside a character class, ignores typed in spaces within the patterns, which would be problematic if a space was part of the pattern to match.  So you could break a working regex if you later decide to add this option. This happened on the Regexlib when the option was first turned on.  A lot of patterns that were written before the switch was flipped suddenly stopped working.

Speaking of .Net when it comes to name groups you can’t use the hex notation to define the group name using the single quote syntax .  However you can avoid any issue with single quotes by using the alternate syntax.