Web Debug

Fix broken web applications, from servers to clients.

Proxy PAC File Tricks and Tips

Dealing with Case Sensitivity


As mentioned above, PAC files are case sensitive. If you are seeing issues with upper/lower case URL’s it is relatively simple to convert everything to lower case at the top of the PAC file and not have to worry about case later on. To do so, simply put this section somewhere near the top of your PAC file:

var lhost = host.toLowerCase();
host = lhost;

<!--more-->

Effective use of Indentations


Another fairly simple programming trick is to make sure you effectively use indentations. It makes your PAC file much easier to read and easier to troubleshoot. The very simple rule says "If you are putting something inside braces ( { } ), indent it one more tab stop. Your close brace should be at the same indent level as the item that opened the brace. The only exception to this rule is that you don’t need to indent your entire PAC file that’s between the "function FindProxyForURL(url, host) {" at the top and the very last closing brace at the bottom. It’s safe to cheat here. Just be sure to indent your IF statements and make things line up nicely for readability. You (and your co-workers) will be happy about this later as they can more easily read through the PAC file.

For a good examples of indenting, see the sample PAC files in the menues to the left.

Dealing with localhost and loopback addresses


Localhost and loopback should always bypass the proxy – Put this near the top of your PAC file.

if ((host == "localhost") ||
(shExpMatch(host, "localhost.")) ||
(host == "127.0.0.1")) {
return "DIRECT";

Safely Blocking Sites at the Browser Instead of the Proxy


Blocking sites is also handy. This can be done for a number of reasons – Spyware/malware sites are very good examples Blocking these sites can be done very easily – Simply return a proxy value somewhere on a loopback address so that the requests never actually leave the local machine to take up network bandwidth. The only caveat with this is to ensure that your selection of port number isn’t actually listening on the PC which could odd behavior.

if (dnsDomainIs(host, ".badspyware.com") ||
dnsDomainIs(host, ".worsespyware2.com")) {
return "PROXY 127.0.0.1:48890";
}

Using alert() for Notifications and Troubleshooting


Alert() can be used very effectively to help troubleshooting browser issues, especially the one mentioned above where the browser occasionally picks the wrong IP for myIpAddress(). For this, pick a bogus hostname – proxyinfo.company.com would work fine. It doesn’t have to be in DNS or registered anywhere – Just placing it in the PAC file is all you need. Be aware, however that the alert message is displayed with EVERY request that matches the condition. If you accidentally get an alert in the wrong place it can be very annoying and render the browser nearly useless. Used properly, however, it can be VERY handy.

if ((host =="proxyinfo.company.com")) {
alert("Local IP address is: " + myIpAddress());
}

Using Variables


Using variables within a PAC file can be an extremely powerful way to simplify your PAC file. You can create variables to hold almost any value - A client's IP address, a proxy server address, a true/false value, etc. This variable can be read and reset at any point in the PAC file.

To use a variable, all you need to do is set it to a value. For example, to set a variable called "myip", enter the following near the top of your PAC file.

myip = myIpAddress();

When setting a variable, you can optionally insert the word "var" in front of it. I typically do not do this, however.

Once the variable has been set, you can the variable name at any further point in the PAC file. For example:

if(isInNet(myip, "192.160.1.0","255.255.255.0")) { ....  }

Why would you do this? In this particular case, it's much "cheaper" computationally to call the function myIpAddress() once and put it into a variable rather than to call it multiple times.

One of most effective uses of varialbles is to set a variable to one piece of data and then change it as your PAC file progresses. I use this for setting a proxy variable and then adjusting it based on client IP address or other conditions. When I get to a RETURN statement I simply return the proxy variable. Here's an example of how that can be used:

function FindProxyForURL(url, host) {
// Set the default proxy variable that users get if they don’t match
// any more specific rule.
proxy = "PROXY coreproxy.company.com:8000";

// Los Angeles WAN subnets go to LA proxy
if (isInNet(myIpAddress(), "10.100.0.0", "255.252.0.0")) {
proxy = "PROXY la-proxy.company.com:8000";
}

// New York WAN subnets go to New York proxy
if (isInNet(myIpAddress(), "10.200.0.0", "255.252.0.0")) {
proxy = "PROXY ny-proxy.company.com:8000";
}


When writing this kind of PAC file, remember to start with the most generic and get to the most specific. If you need to route a single /24 to a specific proxy, do it AFTER you’re done routing the /16’s. Pay attention to where the variable gets set and make sure it doesn’t get overwritten later.

When you have reached a point in your PAC file that you need to return a proxy value, you just return the variable, as shown below.

return proxy;

Safely using IsInNet(host, …)


It is very useful to use isInNet(host., ….), especially when the host is an IP address and you’re trying to match it. For example, you might need to send all traffic in the 10.0.0.0/8 and 192.168.0.0/16 address spaces browser direct but all other direct IP’s via the proxy. Unfortunately, just using IsInNet(host, …) alone causes problems, discussed the Lessons Learned article.

Fortunately, there is a way around it: Write an IF statement that uses a regex check to see if the host is an IP address then use IsInNet(host, …) to check to see if it’s in a specific subnet. Example:

reip = /^\d+.\d+.\d+.\d+$/g;
if (reip.test(host)) {
if (isInNet(host, "10.0.0.0", "255.0.0.0") ||
isInNet(host, "192.168.0.0", "255.255.0.0")) {
return "DIRECT";
}
}

Some have reported issues with later versions of IE not working properly with this type of regex test, but that shExpMatch works properly.

if (shExpMatch(host, "/^\d+.\d+.\d+.\d+$/g")) {
if (isInNet(host, "10.0.0.0", "255.0.0.0") ||
isInNet(host, "192.168.0.0", "255.255.0.0")) {
return "DIRECT";
}
}

You can also use simple regex matches for your IP space, but this is far more elegant and effective, especially if you’re testing multiple subnets.

Using Substrings


It is possible to check only a specified range of characters within the URL or within the host variable. This is referred to as a substring. It can be extremely handy for many things. One of the most common ways to use a substring is to check what protocol it is. This might be handy, for example, if you have a configuration where you have a separate proxy for different protocols – i.e. HTTP requests go to the main company proxy, ftp requests go to the FTP relay and MMS links go to the streaming infrastructure. One important note on substrings - Because of IE's Automatic Proxy Results Cache (See the Lessons Learned section for details) the browser only queries the PAC file once per host and caches that result. Be sure to consider this when using substrings.

Using substrings is fairly simple. The syntax is “varname.substring(begin,end)” where varname is the text-based variable (i.e. host or url) that you want to check, begin is where the substring starts and end is where it finishes. Keep in mind that the first character in a string is position #0, not #1 as you might expect.

Examples:

if (url.substring(0,4) == "http") { return "PROXY http-proxy.company.com:8000"; }  //matches HTTP and HTTPS URLs
if (url.substring(0,3) == "ftp") { return "PROXY ftp-proxy.company.com:8000"; } //matches FTP:// links
if (url.substring(0,3) == "mms") { return "PROXY http-proxy.company.com:8000"; } //matches MMS links

This is only one way to use substrings – If you can find a consistent character position within the URL you can split it out and look at it.

Advanced JavaScript Ideas


This guide is focused on the things that I’ve found are useful when writing PAC files. That isn’t to say that it covers everything you might need to do. JavaScript is a complex language with a lot of very powerful features. You can use nearly all of them within a PAC file. Don’t let this (or any other such guide) limit you to what you can try. Dig into a JavaScript reference and experiment a bit with some of the more advanced features. Think about some of the string manipulation features (length, used in combination with substring, for example), using arrays, writing your own reusable subroutines within the PAC file, etc.

Here’s a non-real-world example to get you thinking about what you could do...

Let’s say your company has fifty office buildings. Each one is assigned a /16 in 10.x – 10.1.0.0/16, 10.2.0.0/16, etc. Each floor gets a range of ten /24 subnets. Floor 1 is .10 -.19, floor 2 is .20-29, etc. In each building, the executives are placed on the 20th floor. So, their subnet range is .200 - .209. An executive in building #13 might be assigned an IP address of 10.13.202.84.

Your team has built a special proxy to be used only be executives, to ensure they get the fastest possible response time. Using basic PAC file functions described so far, you’d have to put in 100+ isInNet(myIpAddress()) lines to match all of these. But there’s a better way – Using the JavaScript split function and arrays. Here’s how..

var myip = myIpAddress();  // Set a variable for the local IP address.
var my-addr-array = myip.split("."); // Split that IP address var into an array.
var mysubnet=parseInt(my-addr-array[2]); // Convert array element #2 into a number and store it in a new variable
if ((mysubnet >= 200) && (mysubnet < = 209)) { // If that number is between 200 and 209//
proxy = "PROXY execproxy.company.com:8000"); // Assign it to the executive proxy
}
else {
proxy = "PROXY proxy.company.com:8000"; // Otherwise they get the standard proxy.
}

Another very useful JavaScript idea is using split. This is a way to break out a string into chunks by splitting on a specific character. The result of this will be an array that you can use containing the split-apart text. A good example of this is to split the client IP address into chunks. For example..

var myip=myIpAddress()
var octects=myip.split(".")

You now have an array variable called "octects" that is the different octects of the IP address string split into an array. You can refer to "octects[0]" for the first octect, "octets[1]" for the second, etc.

One important item to keep in mind is that a string value and a number value are very different. The character "3" stored in text is just that - A string character. The number 3 is a real number that can be manipulated. In the above case, for a IP address of 192.168.103.93 you will have an array with four text strings. "192", "168", "103" and "93". These can be used for an exact match, but not to compare or run any math. You can, however, convert them to numbers using the ParseInt() function. Here's an example:

firstoctet = ParseInt(octects[0]);

Now, the variable "firstoctet" will contain the number 192, not the string "192". This new variable can use any type of math function, greater/less than compares, etc.
These are just a few example – If you can think of something you need to do, it’s probably possible to do it. Need to block all “.pif” files? Use the url.length-3 as the beginning of your url.substring and url.length as the end of it – Is the substring “pif”? If so, send it to a loopback address, like a spyware block. Don’t be afraid to dig in and experiment.

It is, of course, important to add the standard caveat – Assume that you’re going to get hit by a bus tomorrow. Make sure that someone else can support your PAC file. Document it, add appropriate comments, etc. and follow the golden rule - Keep It Simple and Supportable! A PAC file is not the place to be writing obfuscated JavaScript!

Proxy Load Balancing within a PAC File


Many organizations have multiple proxy servers in the same location without a hardware load balancer to distribute traffic. This section provides a method to load balance traffic between two proxies by looking at the last octect of the users' IP address. Addresses with even numbers go to one proxy, addresses with odd numbers go to another. This includes failover, should one proxy become non-responsive. PAC-file based proxy failover can have it's challenges (as noted in the Lessons Learned section) but is perfectly acceptable to use here.

This section is based on a document by a Novell BorderManager guru named Shawn Pond and is used with his permission. It is simple, logical and easy to implement. His original document can be found at www.novell.com/coolsolutions/feature/7949.html.

First, the PAC file code:

// Find the 4th octet
var myip=myIpAddress()
var ipbits=myip.split(".")
var myseg=parseInt(ipbits[3])

// Check to see if the 4th octect is even or odd
if (myseg==Math.floor(myseg/2)
2) {
// Even
proxy = "PROXY p1.company.com:8080; PROXY p2.company.com:8080";
}
else {
// Odd
proxy = "PROXY p2.company.com:8080; PROXY p1.company.com:8080";
}


In this code, the first thign we do is to find the users IP address into a variable called "myip". We then split the address into an array called "ipbits", separating by the period character. We find convert the 4th octect (ipbits[3]) and convert it to a numeric value which is stored in the variable "myseg":

var myip=myIpAddress()
var ipbits=myip.split(".")
var myseg=parseInt(ipbits[3])

To check of even/odd we divide it by the last octect by two, discarding any remainder, then multiply that result by two. If the result is the same as the original 4th octet then it is even. If the number is different then the last octect is odd. If it's even, we set the variable called "proxy" to be the proper proxies to use - P1 first, then P2 as a failover. If it's odd, we set the "proxy" variable to the proper value for odd-number users - P2 as first and P1 as failover.

// Check to see if the 4th octect is even or odd
if (myseg==Math.floor(myseg/2)*2) {
// Even
proxy = "PROXY p1.company.com:8080; PROXY p2.company.com:8080";
}
else {
// Odd
proxy = "PROXY p2.company.com:8080; PROXY p1.company.com:8080";}

Add the rest of your PAC file code. When you have reached a point in your PAC file that you need to return a proxy value, you just use "return proxy;" to send back the proper proxy value for the user.

Original link

http://www.proxypacfiles.com/proxypac/index.php?option=com_content&view=article&id=54&Itemid=83

FindProxyForURL does not work for different urls on same web server

You may find your original auto proxy configuration script (wpad or pac file) is not working for different urls on same web server using shExpMatch in IE9+. This is because a performance enhancement named Automatic Proxy Result Cache are introduced.

Browser Connection Limit

HTTP 1.1 RFC 2616


As RFC2616 recommended,

Clients that use persistent connections SHOULD limit the number of simultaneous connections that they maintain to a given server. A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy. A proxy SHOULD use up to 2*N connections to another server or proxy, where N is the number of simultaneously active users. These guidelines are intended to improve HTTP response times and avoid congestion.

This means the client should control the connections on a per host basis. If  a website gets resource from multiple hosts (www.host1.com, www.host2.com, www.host3.com ..), the client can open 2 connections for each host name.

<!--more-->

Yahoo website performance rules "Minimize Http Requests" are also recommending website should minimize http requests to server as connections limit might caused resource downloading pending on get connections.

Browser Configuration


Earlier versions of browsers strictly followed this standards and soon they find 2 connections per host are not efficient enough for modern web applications. This link provides testing result for connection limits on different versions of browsers.

connections

Different browsers has different settings to control the this limit.

IE can refer to KB28242, this setting applies to IE6 - IE9, while IE10/11 can use the following registry key to set the limit, range from 2-128.

HKLM\SOFTWARE<b>Wow6432Node\Microsoft\Internet Explorer\Main\FeatureControl\FEATUREMAXCONNECTIONSPERSERVER

In Firefox these values are controlled by the network.http.max-persistent-connections-per-server and network.http.max-connections-per-server settings in about:config.

For Chrome it looks this feature are still <a href="http://code.google.com/p/chromium/issues/detail?id=85323" target="blank">under discussion.

Effect of Proxies


Note that if you’re behind a proxy (at work, etc.) your download characteristics change. If web clients behind a proxy issued too many simultaneous requests an intelligent web server might interpret that as a DoS attack and block that IP address. Browser developers are aware of this issue and throttle back the number of open connections.

In Firefox the network.http.max-persistent-connections-per-proxy setting has a default value of 4. If you try the Max Connections test page while behind a proxy it loads painfully slowly opening no more than 4 connections at a time to download 180 images. IE8 drops back to 2 connections per server when it’s behind a proxy, so loading the Max Connections test page shows an upperbound of 60 open connections. Keep this in mind if you’re comparing notes with others – if you’re at home and they’re at work you might be seeing different behavior because of a proxy in the middle.

Testing


Steve Souders provides a max-connection testing page to test this behavior.

http://stevesouders.com/hpws/max-connections.php

I created a testing page for clients to test what is the connection limit for clients.

http://webdebug.net/toolbox/max-connections.html

How Browsers Work

Preface


This comprehensive primer on the internal operations of WebKit and Gecko is the result of much research done by Israeli developer Tali Garsiel. Over a few years, she reviewed all the published data about browser internals (see Resources) and spent a lot of time reading web browser source code. She wrote:

In the years of IE 90% dominance there was nothing much to do but regard the browser as a "black box", but now, with open source browsers having more than half of the usage share, it's a good time to take a peek under the engine's hood and see what's inside a web browser. Well, what's inside are millions of C++ lines...


Tali published her research on her site, but we knew it deserved a larger audience, so we've cleaned it up and republished it here.As a web developer, learning the internals of browser operations helps you make better decisions and know the justifications behind development best practices. While this is a rather lengthy document, we recommend you spend some time digging in; we guarantee you'll be glad you did. Paul Irish, Chrome Developer Relations

This article has been translated into Korean by the community. HTML5 Rocks hosts the German, Spanish, Japanese, Portuguese, Russian and Simplified Chinese versions.

You can also watch Tali Garsiel give a talk on this topic on Video.


How to add managed module build on .NET 4.0 in IIS7

Basic steps to add managed module to IIS,

1. Write the code you want to run on every request, as a .net 3.5 Class Library.
2. Compile it as a strong-named assembly.
3. Install the assembly in the GAC.
4. In IIS7 manager, select the server name in connections, click Modules, click "Add managed module" in actions.
5. Write a name for the module and choose your newly installed assembly in the type dropdown.
6. Make sure the site uses an application pool running in integrated mode.

 

One question remains though!

There are now two GAC's, Microsoft.NET for .net 4.0, and Windows GAC for pre .net 4.0. Because I created my assembly in .net 3.5, it was installed in Windows GAC, and therefore it was avalable in the type dropdown in IIS manager.

When I created my assembly in .net 4.0, it was installed in the Microsoft.NET GAC, and as a result, it was NOT avalable in the type dropdown in IIS manager.

After working with this for a bit, I managed to make it work. What I had to do was:

1. Create the .net 4.0 Class Library, and compile it as a strong named assembly

2. Install it in the .net 4.0 GAC by using the gacutil, located in Program Files (x86)\Microsoft SDKs\Windows\v7.0A\bin\NETFX 4.0 Tools

(Or make Visual Studio compile, sign and install the assembly automatically)

3. Add this line under in applicationHost.config: (it has to be done manually, it can't be done in the manager)


http://stackoverflow.com/questions/9210550/module-registered-in-iis7-doenst-work

Fork me on GitHub