newspaint

Documenting Problems That Were Difficult To Find The Answer To

Category Archives: Java

Jstack Unable to Contact Target Process

In Java one of the most useful tools for diagnosing thread-related issues is jstack. It outputs a stack trace of every thread of a running process – and can also give an indication of the state of locks.

However I was getting the following error message when trying to attach jstack to a running process:

17840: Unable to open socket file: target process not responding or HotSpot VM not loaded
The -F option can be used when the target process is not responding

The solution, amazingly, was to run the jstack command as the same user as the running Java process.

The command to use for this is:

sudo -u <user> jstack -l <pid>

e.g. if the Java process is running as user “oracle” with process ID 45132 then you would execute:

sudo -u oracle jstack -l 45132

JMeter – User Parameters or User Defined Variables?

You almost always want User Parameters because these are recomputed for every thread and, optionally, every iteration. Whereas User Defined Variables are processed at the start and copied to all threads.

A Basic Ant Build Script For Simple Java Projects

Here is a basic Apache Ant build script I use for taking one or more source Java files and associated jars and bundling them all into a single jar with a main class.

I have the following directory structure:

|--build.xml
|--src\
|  +--MyMainClass.java
|  +--OtherClass.java
+--lib\
   +--dependency.jar

When I run ant jar using the below build.xml it will then create new directories:

|--build
|  +--classes
|     +--MyMainClass.class
|     +--OtherClass.class
+--jar
   +--myproject.jar

Running ant clean will remove the newly created folders build/ and jar/.

This tutorial is also very helpful.

<project name="myproject">
  <property name="src.dir"     value="src"/>

  <property name="build.dir"   value="build"/>
  <property name="classes.dir" value="${build.dir}/classes"/>
  <property name="jar.dir"     value="jar"/>
  <property name="jar.file"    value="${jar.dir}/${ant.project.name}.jar" />

  <property name="lib.dir"     value="lib" />

  <property name="main-class"  value="MyMainClass" />

  <path id="classpath">
    <fileset dir="${lib.dir}" includes="**/*.jar" />
  </path>


  <target name="clean">
    <delete dir="${build.dir}" />
    <delete dir="${jar.dir}" />
  </target>

  <target name="compile">
    <mkdir dir="${classes.dir}" />
    <javac srcdir="${src.dir}" destdir="${classes.dir}" classpathref="classpath" />
  </target>

  <target name="jar" depends="compile">
    <mkdir dir="${jar.dir}" />
    <jar destfile="${jar.file}" basedir="${build.dir}">
      <manifest>
        <attribute name="Main-Class" value="${main-class}" />
      </manifest>
      <fileset dir="${classes.dir}" includes="**/*.class" />
      <fileset dir="${src.dir}" includes="**/*.java" />
      <zipgroupfileset dir="${lib.dir}" includes="**/*.jar" />
    </jar>
  </target>
</project>

If I want debugging enabled (see also Ant javac parameters for more options):

  <target name="compile">
    <mkdir dir="${classes.dir}" />
    <javac srcdir="${src.dir}" destdir="${classes.dir}" classpathref="classpath" debug="true" />
  </target>

Looking Inside A Java Jar File

Here are some simple scripts for inspecting a .jar file.

Listing All Classes and Methods in a Jar

I have found this very useful when given a .jar without any other documentation and have been expected to use it to interface to a proprietary system. Being given the classes and methods along with the method signatures (including expected parameter types) has been a life-saver.

#!/bin/bash

JAR_TO_INTERROGATE=$1
if [ -z "$JAR_TO_INTERROGATE" ]; then
  echo "Please provide JAR file name to interrogate"
  exit 1
fi

CLASSES_OF_JAR=$(jar tf "$JAR_TO_INTERROGATE" |grep class |sed 's/.class//g')

javap -classpath "$JAR_TO_INTERROGATE" -s $CLASSES_OF_JAR

This utility can be used as follows:

user@host:~$ interrogate_jar.sh junit-4.10.jar
Compiled from "ActiveTestSuite.java"
class junit.extensions.ActiveTestSuite$1 extends java.lang.Thread{
final junit.framework.Test val$test;
  Signature: Ljunit/framework/Test;
final junit.framework.TestResult val$result;
  Signature: Ljunit/framework/TestResult;
final junit.extensions.ActiveTestSuite this$0;
  Signature: Ljunit/extensions/ActiveTestSuite;
junit.extensions.ActiveTestSuite$1(junit.extensions.ActiveTestSuite, junit.framework.Test, junit.framework.TestResult);
  Signature: (Ljunit/extensions/ActiveTestSuite;Ljunit/framework/Test;Ljunit/framework/TestResult;)V
public void run();
  Signature: ()V
}

Compiled from "ActiveTestSuite.java"
public class junit.extensions.ActiveTestSuite extends junit.framework.TestSuite{
public junit.extensions.ActiveTestSuite();
  Signature: ()V
public junit.extensions.ActiveTestSuite(java.lang.Class);
  Signature: (Ljava/lang/Class;)V
public junit.extensions.ActiveTestSuite(java.lang.String);
  Signature: (Ljava/lang/String;)V
public junit.extensions.ActiveTestSuite(java.lang.Class, java.lang.String);
  Signature: (Ljava/lang/Class;Ljava/lang/String;)V
public void run(junit.framework.TestResult);
  Signature: (Ljunit/framework/TestResult;)V
public void runTest(junit.framework.Test, junit.framework.TestResult);
  Signature: (Ljunit/framework/Test;Ljunit/framework/TestResult;)V
synchronized void waitUntilFinished();
  Signature: ()V
public synchronized void runFinished();
  Signature: ()V
}
  ...

Showing the Java Disassembly for all Methods in a Jar

#!/bin/bash

JAR_TO_INTERROGATE=$1
if [ -z "$JAR_TO_INTERROGATE" ]; then
  echo "Please provide JAR file name to interrogate"
  exit 1
fi

CLASSES_OF_JAR=$(jar tf "$JAR_TO_INTERROGATE" |grep class |sed 's/.class//g')

javap -classpath "$JAR_TO_INTERROGATE" -c $CLASSES_OF_JAR

This utility can be used as follows:

user@host:~$ disassemble_jar.sh junit-4.10.jar
Compiled from "ActiveTestSuite.java"
class junit.extensions.ActiveTestSuite$1 extends java.lang.Thread{
final junit.framework.Test val$test;

final junit.framework.TestResult val$result;

final junit.extensions.ActiveTestSuite this$0;

junit.extensions.ActiveTestSuite$1(junit.extensions.ActiveTestSuite, junit.framework.Test, junit.framework.TestResult);
  Code:
   0:	aload_0
   1:	aload_1
   2:	putfield	#1; //Field this$0:Ljunit/extensions/ActiveTestSuite;
   5:	aload_0
   6:	aload_2
   7:	putfield	#2; //Field val$test:Ljunit/framework/Test;
   10:	aload_0
   11:	aload_3
   12:	putfield	#3; //Field val$result:Ljunit/framework/TestResult;
   15:	aload_0
  ...

JMeter Could Not Initialize MenuFactory Class

I was getting the following messages when starting JMeter:

Uncaught Exception java.lang.NoSuchMethodError: kg.apc.jmeter.gui.GuiBuilderHelper.strechButtonToComponent(Ljavax/swing/JComponent;Ljavax/swing/JButton;)V. See log file for details.
Uncaught Exception java.lang.NoClassDefFoundError: Could not initialize class org.apache.jmeter.gui.util.MenuFactory. See log file for details.

In addition my Edit menu was greyed out in the JMeter window:

JMeter Edit Menu Greyed Out

JMeter Edit Menu Greyed Out

The problem was that I had been placing a custom written Java sampler into the lib/ext/ subdirectory and that plugin was including a conflicting .jar in its .jar (you can put .jars inside .jars).

The solution? Remove the following file from the .jar plugin I created:

  • JMeterPlugins.jar

Enterprise Reasons Why Java Is Not Ready

Java is not an enterprise-ready language. That’s a big call to make considering it has been around for the best part of a decade and is one of the most popular languages in use across all industries – including financial, media, telecommunications.

Java is lacking serious and basic features. They include:

  • no unsigned integer support – according to Java language designer James Gosling, “Quiz any C developer about unsigned, and pretty soon you discover that almost no C developers actually understand what goes on with unsigned, what unsigned arithmetic is”. Personally I think signed arithmetic is hard – because you need to find out what happens to the most-significant bit when you left-shift. The lack of unsigned arithmetic has stifled and frustrated many a network developer.
  • no conditional compilation support – I have two classes I want to link against for different situations. But one class has support for an overridden method, the other doesn’t. There’s no simple way for my dependent class to quickly include or exclude the methods in question.
  • hard limit of 64KB on method size – no normal subroutine will ever exceed 64KB in size so this limitation is rarely encountered by developers. But when you write a code generator to emulate captured code you quickly find yourself hitting this limit and are forced to introduce complexity by breaking up your generated code into smaller methods.
  • hard limit of 64KB on automatic variable count – while a tree or hashmap can contain megabytes of data you may not declare too many objects in your method or you will hit this 64KB limit.

Java also makes life unnecessarily difficult:

  • forcing exceptions to be handled – this results in extremely verbose code – the try .. catch blocks can often bloat code by a factor of 300+% and this is unacceptable when unnecessary. Exception handling is useful, being forced to use it is wrong.
  • refusing inner scope local variables with the same name as one in an outer scope – in an effort to protect inexperienced developers from making simple mistakes more advanced developers are stopped from re-using utility variable names in an inner scope with the same name as a variable used in an outer scope. This should be an optional warning at worst rather than a compile-time error.

JMeter Error in TestPlan See Log File

Today I had a new error in JMeter. I tried to open a test plan (.jmx file) that I’d saved a few minutes earlier before re-starting JMeter. The error said “Error in TestPlan – see log file”.

Error in TestPlan - see log file

Error in TestPlan – see log file

I opened up jmeter.log to have a look. It contained:

2013/08/15 14:07:45 INFO  - jmeter.gui.action.Load: Loading file: /home/user/data/work/jmeter/my-instance/2013-08-15-login.jmx 
2013/08/15 14:07:45 INFO  - jmeter.services.FileServer: Set new base='/home/user/data/work/jmeter/my-instance' 
2013/08/15 14:07:46 INFO  - jmeter.save.SaveService: Testplan (JMX) version: 2.2. Testlog (JTL) version: 2.2 
2013/08/15 14:07:46 INFO  - jmeter.save.SaveService: Using SaveService properties file encoding UTF-8 
2013/08/15 14:07:46 INFO  - jmeter.save.SaveService: Using SaveService properties file version 1382180 
2013/08/15 14:07:46 INFO  - jmeter.save.SaveService: Using SaveService properties version 2.3 
2013/08/15 14:07:46 INFO  - jmeter.save.SaveService: All converter versions present and correct 
2013/08/15 14:07:46 ERROR - jmeter.save.SaveService: Conversion error com.thoughtworks.xstream.converters.ConversionException: kg.apc.jmeter.vizualizers.CorrectedResultCollector : kg.apc.jmeter.vizualizers.CorrectedResultCollector
---- Debugging information ----
message             : kg.apc.jmeter.vizualizers.CorrectedResultCollector
cause-exception     : com.thoughtworks.xstream.mapper.CannotResolveClassException
cause-message       : kg.apc.jmeter.vizualizers.CorrectedResultCollector
class               : org.apache.jorphan.collections.ListedHashTree
required-type       : org.apache.jorphan.collections.ListedHashTree
converter-type      : org.apache.jmeter.save.converters.HashTreeConverter
path                : /jmeterTestPlan/hashTree/hashTree/kg.apc.jmeter.vizualizers.CorrectedResultCollector
line number         : 252
------------------------------- : kg.apc.jmeter.vizualizers.CorrectedResultCollector : kg.apc.jmeter.vizualizers.CorrectedResultCollector

This error happened immediately after I did an apt-get dist-upgrade so I wasn’t sure what had happened.

In my case a required class had gone missing – in this case it was the visualisers from jmeter-plugins.org. I downloaded a new set and put the file JMeterPlugins-Standard.jar in the lib/ext/ subdirectory.

This fixed the problem. I was able to re-start JMeter and load my test plan successfully.

Summary: a dependent class that the test plan relied on was missing from the plugins directory.

Why Won’t My JMeter Java Sampler Class Load?

Infuriating! Having build a Java Sampler for JMeter and having placed it in the lib/ext/ folder I start up JMeter but when I go to add a Java Sampler my custom class isn’t listed as one of those available!

It seems that JMeter is silently failing to load my JAR file.

So what is going on? Clearly we need to make JMeter more verbose. Time to create a properties file. Try this for measure:

log_level.jmeter=DEBUG
log_level.jmeter.junit=DEBUG
log_level.jmeter.control=DEBUG
log_level.jmeter.testbeans=DEBUG
log_level.jmeter.engine=DEBUG
log_level.jmeter.threads=DEBUG
log_level.jmeter.gui=WARN
log_level.jmeter.testelement=DEBUG
log_level.jmeter.util=DEBUG
log_level.jmeter.util.classfinder=DEBUG
log_level.jmeter.test=DEBUG
log_level.jmeter.protocol.http=DEBUG
# For CookieManager, AuthManager etc:
log_level.jmeter.protocol.http.control=DEBUG
log_level.jmeter.protocol.ftp=WARN
log_level.jmeter.protocol.jdbc=DEBUG
log_level.jmeter.protocol.java=WARN
log_level.jmeter.testelements.property=DEBUG
log_level.jorphan=DEBUG

log_file=jmeter-debug.log

Save this as “myprops.txt” (or whatever you like) and then issue the command:

bin/jmeter -p myprops.txt

Now, have a look in the newly created jmeter-debug.log file for the evidence!

2013/02/05 13:53:41 DEBUG - jorphan.reflect.ClassFinder: MyJMeterClass/src/JMeterAction01StartClient : Unsupported major.minor version 51.0

This indicates that the JAR was compiled using the JDK version 7 but I’m trying to run this using the version 6 JRE.

Using the JMeter BeanShell Sampler to Write to the Response Data Tab of the Results Tree

It may be desirable to write information to the “Response data” tab of a BeanShell sampler’s result in the “View Results Tree” listener in JMeter.

This can be achieved using the basic code as an example:

String output = "";
output += "hello\n"; // append "hello" and a newline
output += "world!\n"; // append "world!" and a newline

SampleResult.setResponseData( output );
SampleResult.setDataType( org.apache.jmeter.samplers.SampleResult.TEXT );

Tested using JMeter 2.8 on Linux.

Other JMeter BeanShell Tricks

Listing All the Thread Context Variables

Quick and Dirty

import java.util.Map;

org.apache.jmeter.threads.JMeterVariables vars = ctx.getVariables();

String output = "";
output += "varcount = " + vars.entrySet().size() + "\n";

// iterate over variables
for ( Map.Entry entry : vars.entrySet() ) {
  output += "  - \"" + entry.getKey() + "\" => \""
    + entry.getValue().toString() + "\"\n";
}

SampleResult.setResponseData( output );
SampleResult.setDataType( org.apache.jmeter.samplers.SampleResult.TEXT );

Note that the type returned by vars.entrySet() is actually a Set<Map.Entry<String,Object>> but BeanShell does not allow the angle brackets to be used. The above, however, is still valid (without the angle brackets).

Keywords Alphabetical Order

The following (tested with JMeter 2.8 BeanShell listener) lists the variables in alphabetical order:

import java.util.Map;

org.apache.jmeter.threads.JMeterVariables vars = ctx.getVariables();

// store text to be displayed in a String variable
StringBuffer output = new StringBuffer( "" );
output.append( "varcount = " + vars.entrySet().size() + "\n" );

// convert to array of keys (so we can sort later)
String[] keys = new String[ vars.entrySet().size() ];
int idx = 0;
for ( Map.Entry entry : vars.entrySet() ) {
  keys[idx] = entry.getKey();
  idx = idx + 1;
}

// sort
java.util.Arrays.sort( keys );

// display
for ( String key : keys ) {
  Object value = vars.getObject( key );
  if ( value != null ) {
    output.append( "  - \"" + key + "\" => \""
      + value.toString() + "\"\n"
    );
  } else {
    output.append( "  - \"" + key + "\" => unknown\n" );
  }
}

SampleResult.setResponseData( output.toString() );
SampleResult.setDataType( org.apache.jmeter.samplers.SampleResult.TEXT );

Creating a Java Sampler for JMeter

This was written while using JMeter versions 2.7 and 2.8. This article was written due to the difficulty I initially found in creating a Java Sampler for JMeter.

A Java Sampler is an easy way of creating a JMeter test by using native Java code. In fact there are several approaches that can be taken when creating a JMeter Sampler using Java code – they include:

  • BeanShell Samplerpros: very quick and dirty, no need to create your own class or compile anything, cons: very limited in power and flexibility
  • JUnit Samplerpros: can specify multiple test types inside one class, cons: cannot provide variables to test from JMeter script
  • Java Samplerpros: can specify JMeter script variables to class, cons: cannot have more than one test in a class

JUnit Sampler

A JUnit Sampler can be quickly constructed using the following as a template:

package com.wordpress.newspaint.jmeter.junit.demo;

import junit.framework.TestCase;

// test methods MUST BEING WITH THE PREFIX "test"
public class ExampleJUnitTestClass extends TestCase {
    float first = (float)2.0;
    float second = (float)20.0;

    public void testaddition() {
        assertTrue( "Addition Greater Than", ( ( first + second ) > 21.99 ) );
        assertTrue( "Addition Less Than", ( ( first + second ) < 22.01 ) );
    }

    public void testmulti() {
        assertTrue( "Multiplication Greater Than", ( ( first * second ) > 39.99 ) );
        assertTrue( "Multiplication Less Than", ( ( first * second ) < 40.01 ) );
    }
}

This can be compiled and the resulting jar placed in the lib/junit subdirectory of the JMeter distribution.

Tests are very simple and apart from duration and pass/fail results there is little more that can be gained using this method.

Java Sampler

The Java Sampler is a powerful means of testing just about anything. The big advantages are that variables can be taken from the JMeter script (such as thread number, or user name, or whatever you want) and that results can contain a specific error code as well as custom messages and data – more importantly you can control when the timer for the test starts and finishes.

The following can be an example Java Sampler class:

package com.wordpress.newspaint.jmeter.java.demo;

import java.io.Serializable;
import org.apache.jmeter.protocol.java.sampler.AbstractJavaSamplerClient;
import org.apache.jmeter.config.Arguments;
import org.apache.jmeter.samplers.SampleResult;
import org.apache.jmeter.protocol.java.sampler.JavaSamplerContext;

public class ExampleJavaTestClass extends AbstractJavaSamplerClient implements Serializable {
    private static final long serialVersionUID = 1L;

    // set up default arguments for the JMeter GUI
    @Override
    public Arguments getDefaultParameters() {
        Arguments defaultParameters = new Arguments();
        defaultParameters.addArgument("URL", "http://www.google.com/");
        defaultParameters.addArgument("SEARCHFOR", "newspaint");
        return defaultParameters;
    }

    @Override
    public SampleResult runTest(JavaSamplerContext context) {
        // pull parameters
        String urlString = context.getParameter( "URL" );
        String searchFor = context.getParameter( "SEARCHFOR" );
	
        SampleResult result = new SampleResult();
        result.sampleStart(); // start stopwatch
		
        try {
            java.net.URL url = new java.net.URL(urlString + "?q=" + searchFor);
            java.net.HttpURLConnection connection = (
                java.net.HttpURLConnection
            )url.openConnection(); // have to cast connection
            connection.setRequestMethod("GET");
            connection.connect();

            result.sampleEnd(); // stop stopwatch
            result.setSuccessful( true );
            result.setResponseMessage( "Successfully performed action" );
            result.setResponseCodeOK(); // 200 code
        } catch (Exception e) {
            result.sampleEnd(); // stop stopwatch
            result.setSuccessful( false );
            result.setResponseMessage( "Exception: " + e );

            // get stack trace as a String to return as document data
            java.io.StringWriter stringWriter = new java.io.StringWriter();
            e.printStackTrace( new java.io.PrintWriter( stringWriter ) );
            result.setResponseData( stringWriter.toString() );
            result.setDataType( org.apache.jmeter.samplers.SampleResult.TEXT );
            result.setResponseCode( "500" );
        }

        return result;
    }
}

This can be compiled and the resulting jar placed in the lib/ext subdirectory of the JMeter distribution.

The ability to set variables can be seen from this resulting Sampler screenshot in JMeter:

Java Sampler Demo Showing Variable Input

Java Sampler Demo Showing Variable Input

Those variables can be set to anything – including dynamic variables such as “${__threadNum}”.

The advantage of setting response data is that it will be displayed in the “View Results Tree” listener’s “Response Data” tab – so I find putting the stack trace of any exceptions in there very useful to assist in debugging my tests. Such an example is shown in the next screen shot when attempting to provide an invalid domain name to the above example:

Stack Trace Displayed in Request Data Tab in Results Tree

Jars

I found I needed the following jars to compile this class:

  • ApacheJMeter_core.jar
  • ApacheJMeter_java.jar

Persistent Variables in a Thread

Maybe you want to create several steps/tests. So you create several classes. But how do you share information between tests? Let’s say your first test returned a certain string you wanted to use as the basis for your second test.

JMeter allows you to store variables in the current Thread’s context – so each thread would have its own context.

import org.apache.jmeter.threads.JMeterContextService;
import org.apache.jmeter.threads.JMeterVariables;
// store a value
JMeterVariables vars = JMeterContextService.getContext().getVariables();
vars.putObject( "mySpecialValue", "http://www.apache.org/" );
JMeterContextService.getContext().setVariables( vars );
// fetch a value
JMeterVariables vars = JMeterContextService.getContext().getVariables();
String mySpecialValue = (String)vars.getObject( "mySpecialValue" );
// delete a value
JMeterVariables vars = JMeterContextService.getContext().getVariables();
vars.remove( "mySpecialValue" );
JMeterContextService.getContext().setVariables( vars );

Note that you can store anything including classes. But if your thread loops the value will persist – so you must explicitly remove it but setting the value to null if you don’t want to use it any more. Overwriting a value will cause the existing to be freed/destroyed.

You can also use the variables you set in other samplers – such as the HTTP Sampler! If you store a String in, say, “mySpecialValue” then you can refer to it in your other samplers using the JMeter variable notation (in this case ${mySpecialValue}).

BeanShell Interpreter

This is a quick-and-dirty means of adding Java code to your JMeter scripts. But best to read JMeter best practices first.

There is another post about the BeanShell Interpreter at this link.