dominoExperts.com - Powered by Domino 8.5.2 Domino Accelerator Pack
- Reduce network traffic
Lotus Triple Search DominoExperts + Blogs + R8 forum
dominoExperts.com -> Script vault -> Java

 GZip servlet to gzip your pages


Reads:   
Tomas NielsenPost date: 2007-08-10 17:25

I have played a little with gzip compression. I am not sure of IBM's status of including it in domino. There are som issues with gzip and Internet Explorer so I can understand IBM's reluctance to support it.

Anyways the servlet is listed below. You can take any domino-resource (in fact any web-resource what so ever) and supply the link as a parameter and the servlet will gzip the contents and deliver it.

Notice that it will retrieve the content using the credentials of the logged in user. It mirrors all the cookies in the request to the server.

This is the url format (you can try the link if you want):

http://www.dominoExperts.com/servlet/gZip?url=/

It supports relative URL's but you could also pick up content from other sites.

I have included the class file for those lazy kids that don't want to compile the servlet.  

/*
* Author: Tomas Nielsen
* Created: 2007-08-09
* www.dominoExperts.com
*//*
Servlet to retrieve a URL's content and deliver it compressed using gzip - if the client can handle it.
The servlet is completely transparent if the user is logged in, the page is retrieved with the users credentials.
This is done by mirroring all cookies trough the proxy.
Note: URL's can NOT contain spaces. That is not a valid URL. %20 or + is the way to go.
Beware that there is a bug in HttpServletRequest.getParameter that will cause some disturbances with encoding if you
use encoded URL's. Depending on your configuration it can for example misinterpret UTF-8 as ISO8859-1 or vice versa
(it does not query the browser settings before converting, and some browsers do not even send encoding, so it is a gamble).
Steps performed:
- Get the URL content
- Determine clients ability to do gzip
- If able do the zipping thing
- Deliver content
Invoked trough:
/servlet/gZip?url=xxx
*/
import
 java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.net.URL;
import java.net.URLConnection;
import java.util.zip.GZIPOutputStream;
import javax.servlet.ServletException;
import javax.servlet.http.Cookie;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
public class gZip extends HttpServlet {
public void doGet(HttpServletRequest req, HttpServletResponse res) throws ServletException, IOException {
        long executionTime = System.currentTimeMillis();
        try {
                String u = req.getParameter("url");
                u = (u.charAt(0)=='/' ? "http://" + req.getServerName() + u : u);  // Allow relative URL's.                URL url = new URL( u );
                URLConnection con;
                con = url.openConnection();
                
// Simulate being the user requesting the original page.                int i = 0;        String c = "";
                Cookie cakes[] = req.getCookies();
                
if (cakes != nullwhile (i < cakes.length ) {
                        c += cakes[i].getName() + "=" + cakes[i].getValue() + "; ";
                        i++;
                }
                con.setRequestProperty (
"Cookie",  c );
                con.connect();
                
if (con == null) {  // Bad connect.                        PrintWriter o = res.getWriter();
                        o.println(
"Error reading url: " + req.getParameter("url"));
                        return;
                }
                String contentType = con.getContentType();
                BufferedReader br;
                br = 
new BufferedReader(new InputStreamReader(con.getInputStream()));
                String line = "";
                String content="";
                // Get the page source.                while((line = br.readLine()) != null) {  // Loop trough all lines.                                content += line + "\n";
                }                br.close();
                
// set the content type to the original content type                res.setContentType(contentType);
                
// See if browser can handle gzip                String encoding=req.getHeader("Accept-Encoding");
                if (encoding!=null && encoding.indexOf("gzip")>=0) {  // gzip broser?                        res.setHeader("Content-Encoding","gzip");
                        OutputStream o=res.getOutputStream();
                        GZIPOutputStream gz=
new GZIPOutputStream(o);
                        gz.write(content.getBytes());
                        gz.close();
                o.close();
                } 
else {  // Some old browser -> give them plain text.                        PrintWriter o = res.getWriter();
                        o.println(content);
                        o.flush();
                        o.close();
                }
        } 
catch(Exception e)
        {
                e.printStackTrace();
        }
}
}


gZip.class
Fredrik StöckelPost date: 2007-08-12 23:00

Very nice and handy! :)  


Bryan KuhnPost date: 2007-08-16 02:19

is there a loss of memory and processor power by using this gzip servlet? And if so does it outweigh the benefits of smaller file sizes.


Tomas NielsenPost date: 2007-08-16 18:33

There is a small price to pay for the proxying and the compression part. But I have had a hard time measuring that performance loss. (Even on a 500MHz machine)

I do not think it will be a concern. The memory overhead is very low.

For smaller files I do not think it is worth to gzip but for larger files especially in a WAN or a LAN under heavy load it would probably do wonders.


YoGiPost date: 2007-10-02 15:36

Hi,

Nice idea !

I've made something similar, it works well in dev environnement, but when in prod, the nhttp task just keep eating up the memory and becoming slower and slower (because there's lots of users,I guess). So finally it's kinda unusable.

Have you experienced such issues with your servlet ?

FYI, I'm running Domino 7.0.2 FP1 on Windows 2003, with the Domino servlet manager, and you can find the source code here :

http://darkmag.net/darkBlog/files/ProxyServlet.java

Best regards,

Yogi


Tomas NielsenPost date: 2007-10-02 21:44

Hi YoGi and welcome here!

I have not found any problems with memory leaks in servlets. The nhttp task do grow in memory size on all servers but it is seldom that I have experienced that it should affect the servers performance. The memory hogging usually stops after a while.

The server versions I have servlets running in production is 6.5 and 7 versions (customer servers with heavy load).

Nice to see someone else working on this! And thank you for the credits in the code!


marky_ukPost date: 2007-10-15 12:29

Hi guys,

This is sweet and can't wait to try it out in the next few days :-)

One quick question - has anyone tried and succeeded at applying these principles to TinyMCE (http://tinymce.moxiecode.com/) - without gzip compression we're talking anywhere between 40 and 90 files or so to be downloaded!  I would love it all in one or two files, but tried and failed thus far :-(

Marky_UK


Tomas NielsenPost date: 2007-10-16 21:42

There exists a JSP solution to GZip the whole of Tiny. That JSP should - in theory - be easy to compile as a servlet. (Look at the download page)

But you would probably get some good performance improvements just by adding a cache rule on the server (2 minutes work).

Look at this post it will get you started with caching in Domino if you are not already doing it.


Tomas NielsenPost date: 2007-12-01 16:45

By request I have added support for Basic authentication (Session authentication: Disabled).

/*
* Author: Tomas Nielsen
* Created: 2007-08-09
* www.dominoExperts.com
*//*
Servlet to retrieve a URL's content and deliver it compressed using gzip - if the client can handle it.
The servlet is completely transparent if the user is logged in, the page is retrieved with the users credentials.
This is done by mirroring all cookies trough the proxy.
Note: URL's can NOT contain spaces. That is not a valid URL. %20 or + is the way to go.

Beware that there is a bug in HttpServletRequest.getParameter that will cause some disturbances with encoding if you
use encoded URL's. Depending on your configuration it can for example misinterpret UTF-8 as ISO8859-1 or vice versa
(it does not query the browser settings before converting, and some browsers do not even send encoding, so it is a gamble).

Steps performed:
- Get the URL content
- Determine clients ability to do gzip
- If able do the zipping thing
- Deliver content

Invoked trough:
/servlet/gZip?url=xxx
 
 * History:
 * 2007-12-01: Added support for basic authentication (Session authentication: Disabled - in Domino)  
 
*/
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.net.URL;
import java.net.URLConnection;
import java.util.zip.GZIPOutputStream;

import javax.servlet.ServletException;
import javax.servlet.http.Cookie;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

public class gZip extends HttpServlet {

public void doGet(HttpServletRequest req, HttpServletResponse resthrows ServletException, IOException {
  try {
    String u = req.getParameter("url");
    u = (u.charAt(0)=='/' "http://" + req.getServerName() + u : u);  // Allow relative URL's.
    URL url = new URL);
    URLConnection con;
    con = url.openConnection();
    // Simulate being the user requesting the original page.
    int i = 0;  String c = "";
    Cookie cakes[] = req.getCookies();
    if (cakes != nullwhile (i < cakes.length ) {
      c += cakesi.getName() "=" + cakesi.getValue() "; ";
      i++;
    }
    con.setRequestProperty ("Cookie",  c );
    
    if (req.getHeader("Authorization"!= null) {  // Add support for Basic Authentication.
      con.setRequestProperty("Authorization", req.getHeader("Authorization"));
    }
    
    con.connect();
    if (con == null) {  // Bad connect.
      PrintWriter o = res.getWriter();
      o.println("Error reading url: " + req.getParameter("url"));
      return;
    }
    if (con.getHeaderField(0).indexOf("401 Unauthorized")>-1) {  // Force the gray login dialog if not logged in.
      res.setStatus(401);
      res.setHeader("WWW-Authenticate""Basic realm=/");
      return;
    }
    String contentType = con.getContentType();
    BufferedReader br;
    br = new BufferedReader(new InputStreamReader(con.getInputStream()));
    String line = "";
    String content="";
    // Get the page source.
    while((line = br.readLine()) != null) {  // Loop trough all lines.
        content += line + "\n";
    }
    br.close();
    
    // set the content type to the original content type
    res.setContentType(contentType);
    // See if browser can handle gzip
    String encoding=req.getHeader("Accept-Encoding");
    if (encoding!=null && encoding.indexOf("gzip")>=0) {  // gzip broser?
      res.setHeader("Content-Encoding","gzip");
      OutputStream o=res.getOutputStream();
      GZIPOutputStream gz=new GZIPOutputStream(o);
      gz.write(content.getBytes());
      gz.close();
          o.close();
    else {  // Some old browser -> give them plain text.
      PrintWriter o = res.getWriter();
      o.println(content);
      o.flush();
      o.close();
    }
  catch(Exception e)
  {
    e.printStackTrace();
  }
}
}

gZip.class
grydanPost date: 2008-04-15 13:34

Can You write a procedure for install your servlet java in a Server Domino for using for a particular WebSite? (For example a site from a blog from template Dominoblog)

Tnx You for your work!


Tomas NielsenPost date: 2008-12-30 01:13

Well there is a problem with Domino not being able to do recursive calls to the HTTP-stack.

The solution I present here run a 1/39 (Default a server has 40 HTTP-threads and the servlet uses one of them) risk of deadlocking its own HTTP-thread. It is fine to prove the point of GZip but I would not run it in a production environment.

I can however recommend looking into Domino Accelerator Pack and get GZip compression with highly optimized C-code speed combined with caching.


rishantPost date: 2013-03-16 19:00
hi Tomas, is it possible compress one folder using gzip in firstServlet and send outputstream in SecondServlet for unzip.



RSS feed
Subscribe to Forum

Share this page

Top posters
Tomas Nielsen212
Joacim Boive27
Fredrik Stöckel27
Danne14
Niklas Waller13
Kenneth Haggman11
Bryan Kuhn10
Daniel Lehtihet9
Jonas Israelsson8
dm997