Tuesday, August 30, 2016

Peculiar problem: Recover from java.lang.OutOfMemoryError

So, a while back, I asked the good people at StackOverflow this question and in a relatively short period of 18hours, I got some vague answers. As always, I always look at other similar questions before posting my own, to gleam some knowledge on the various ways the problem can occur, and to find a nugget of information that I might find against my own specific problem. Alas, it seems like I had overlooked one in my state of half-sleep last day. Its after a good three hours of sleep that I arrive at my solution - but ofcourse, someone else had to beat me to it... by posting 45seconds earlier than when I intended to post my answer :P
Okay, enough chit-chat, here's the code from the question:
public class SpeedDemoClass {
        static int iterations=0;
        static int incrementor=10000000;
    public static void main(String[] args) {
        while(incrementor>1){
            try{
              iterations=iterations+incrementor;
              iterator(iterations);
            }
            catch(Exception e){
                System.out.println("So this is the limiting error: "+e);
                iterations=iterations-incrementor;
                incrementor=incrementor/10;
                iterations=iterations+incrementor;
            }
        }
        System.out.println("The upper limit of iterations is: "+iterations);
    }

    public static void iterator(int a){
            long start_time= System.currentTimeMillis();
            StringBuilder sbuild= new StringBuilder("JAVA");
            for(int i=0;i<a;i++){
                sbuild.append("JAVA");
            }
            System.out.println("Performing "+a+" append operations;"
                    +"process completed in :"
                    +(System.currentTimeMillis()-start_time)+"ms");
    }
}
directly, Jump to the solution
With my manual tests, I was fascinated by the number 92,274,686 that returned to me as the upper limit of iterations that could occur before the java.lang.OOME occured. That's the magic number; I called it the magic number. I didi't know why it was so, and whether or not it was that value for my own computer only. This information was kindly provided by one SO participant in my question:
Your PC probably has 8 GB of RAM? Try using java -Xmx7500m. You may be able to go 3x as far. Because by default, Java only uses 25% of your memory simply to allow other processes to use some memory, too. If you want Java to use all your memory, you have to tell it that this is desired.

I was basically trying to achieve the following things:
  1. Take a counter/var iterations (say I) and a increment value/var incrementor (say X).
  2. So there exists an arbitrary upper limit of the number of iterations that can happen for a certain operation. Here we chose the string append operation for StringBuilder()
  3. As I mentioned earlier, In my earlier manual tests, I had already determined the number to be 92,274,686 for this operation. The task that I undertook was to find that number programmatically.
  4. A simplest logical way is to introduce an impossible number (say, 100,000,000) that will surely cause an error. Reverse logic is to iterate upto that number. We do this by adding X to I in each pass.
  5. In the first pass, X=10million. So, I=I+X produces I=10,000,000 which is below the upper limit and hence usable.
  6. After each pass, the operation I=I+X compounds 10,000,000 more iterations to its previous value
  7. After reaching an impossible value, like in forward logic(4), the program is supposed to backtrack to the last possible state and then decrease X to 1/10th of its value.
  8. The whole operation is continued with 1/10th of previous value of X until there is another error encountered like (4) and (7); upon which the new value of X is again operated to 1/10th its value.
  9. The whole operation continues as usual with each successive new values of X while X>0. This indicates two-fold (a) There is nothing more to increment beyond this point (b) The upper bound has reached after the final increment value of I with X encounters and error.
  10. This way, the upper limit can be reached.
Ofcourse, it was not to be that easy! It promptly showed me an error that I pondered over for way too long, in my opinion. Sometimes, the solutions are just so obvious and simple but you can't see that right-at-your-face at the moment. This was one of those times. Ofcourse, I followed some advice and got more confused.

The error it produced at the output was as follows:

It's 9:45PM and a friend asks to see this question, and with half-a-mind I give the link to my SO Question about this. With nothing else to do, I begin reading my own question and stumble upon a solution in a related one. He does the same and just as I'm about to say I've got it, I get a response "bro milyo?"("Did you figure it out bro?"). I had infact arrived at the same conclusion minutes ago and was preparing an SO Answer. He beat me to it by 45seconds. What are the chances? :D

Here's the solution:
public class SpeedDemoClass {
    static int iterations=0;
    static int incrementor=100000000;
    public static void main(String[] args) {
        while(incrementor>0){
            try{
                iterations=iterations+incrementor;
                int a = iterations;
                long start_time= System.currentTimeMillis();
                StringBuilder sbuild= new StringBuilder("JAVA");
                for(int i=0;i<a;i++){
                    sbuild.append("JAVA");
                }
                System.out.println("Performing "+a+" append operations;"
                    +"process completed in :"
                    +(System.currentTimeMillis()-start_time)+"ms");
            }
            catch(OutOfMemoryError e){
                System.out.println("OutOfMemory bound reached beyond this point with error: "+e
                +"\nReverting back, and Changing the value of incrementor by 1/10th...");
                iterations=iterations-incrementor;
                incrementor=incrementor/10;
                iterations=iterations+incrementor;
            }
        }
        System.out.println("The upper limit of iterations is: "+iterations);
    }
}

OUTPUT:

0 comments

Post a Comment