Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
AWS Route 53 Health Check
#1
What's the best way to check the health of an Intraweb application?

Basically, on AWS' Route 53 I created a health check that connects to my server, requests http://myserver/health-check and if gets back OK, my server is up (the health check is looking for "OK" string to decide if the server is up or not).
My server went down and the health check didn't alarm me. This only happened once, but it made me realize that my solution is not the best.
My original way of checking the sever was like this: In IWServerControllerBaseNewSession I have this piece of code:


Code:
[...] else if ContainsText(WebApplication.Request.PathInfo, 'health-check') or ContainsText(WebApplication.Request.Query, 'health-check') then
  begin
    // respond with OK and don't create a session
    WebApplication.Response.WriteString('OK');
    WebApplication.Terminate;
  end

The problem: One of the servers run out of available connections in the firedac connection pool and the server failed to create new sessions (not sure yet why). The IWError.html page was shown to all users that tried to connect to the server, but the http://myserver/health-check was still returning OK, so it didn't trigger the failed health check. 

I could configure the health check to look for a string in the login page, but the login page returned by intraweb creates a session and considering that the route 53 makes a request every 10 seconds or faster, I don't think it would be a good idea. 

My question is, do any of you have health checks running on a intraweb application? How did you set it up?

TIA
Reply
#2
HI ioan,

I have several endpoints that my health monitor can call.  The main one is
http://beta.governmenttools.com/CrashMag...#36;LOADED
I can also call $SYSCONNECTION or $SYSTABLE to do a more thorough test.  I currently only check $LOADED. I get a text when that returns FALSE.

I respond to the call from the ServerController -> BaseBeforeDispatch().  This avoids having a session created. (it also means that it checks for this specific request on every call - but that's pretty fast - just looking for '$LOADED')
```
if pos( '$LOADED', uppercase( aRequest.PathInfo )) > 0 then
   begin
     aResponse.WriteString( pdmisc.iif( Serving, 'TRUE', 'FALSE' ));
     aResponse.Expires := now;
     Handled := true;
   end
```

When I upgraded to IW 15, I started moving things over to CustomContent handlers.  So I've also got this:
http://beta.governmenttools.com/CrashMag...36;Version
If I understand it correctly, I've specified that a session is not needed for this call. I'm not exactly sure when custom handlers get checked and how they affect sessions if they don't need one.  I don't worry too much about spinning up new sessions as long as they are destroyed when done.  I also require a login to get a "real" session - and that's the more heavy-weight session.

The nice thing about any of these solutions is that you're not just testing for a response, but you can do any processing you like before responding.  You can even return different results if your monitor supports it.

Hope that helps.
Pete
Reply
#3
Hi Ioan,

I use a Custom Content handler to create a response that consists of a list of active sessions, and I also keep up with "cumulative" and "peak number" of active sessions (added variables in ServerController, updated in ...BaseNewSession).

You can do a quick scan of sessions to rip out info that you might find important, so long as you don't do it too often and load things up.

The info you return would depend on what you decide you need to determine "health".

Dan
Reply
#4
Thank you both. Yes, it seems that Custom Content Handlers is the way to go. I'll change my code to use this feature.
Reply
#5
Hello,

I have an application deployed on AWS using Elastic Load Balancer (also for SSL offloading) and implemented health check for EC2 LB using a simple custom content handler for this If my content handler hits an exception, I have it send me an email (via Amamzon simple email service). I started out just using EC2 LB to offload SSL, then I updated to have the one EC2 host running 2 instances of app and it works great. I need to setup AWS based alert monitoring next to make sure things are up and receive notifications.
Reply


Forum Jump:


Users browsing this thread: 2 Guest(s)