System.loadLibary()

The parameter to System.loadLibrary() should be the name of the libary, without path, extension or "lib":

System.loadLibrary("payment");

Depending on your platform, this will load libpayment.so or payment.dll.

Although it is not an error to call System.loadLibrary() multiple times, loading the libraries is often done in a static code block:

class Test {
    static {
        System.loadLibrary("payment");
    }
}

java.library.path

Java searches several directories for this libary file. The directories to search are specified in the property java.library.path. On Linux, this is a copy of the environment variable LD_LIBRARY_PATH. On Windows, it is a copy of the PATH variable.

Although changing the java.library.path has influence on how Java loads its libraries, it has no effect on Windows. When your library depends on other DLLs, these are loaded without taking java.library.path into account. If the library you load needs another library, which can not be found, you get an UnsatisfiedLinkError with the message "Can't find dependent libraries". To solve this, you can load the dependant libraries yourself using multiple calls to System.loadLibrary(), loading all libaries one by one.

When the library can not be found, you get a java.lang.UnsatisfiedLinkError with the message "no library in java.library.path". To solve this, move the library in one of the directories specified by java.library.path or alter the property so that it points to the directory containing your library. (read how to do this in Maven). You can print this property with the following code:

System.out.println(System.getProperty("java.library.path"));

Method decoration

Another possibility when things do not work is when the DLL can be found, but the methods can not. A DLL exports functions according to specific names an calling conventions. When your compiler is not configured the right way your methods may get another name than they should be. To solve this, you should include the header file generated by javah and configure your compiler the right way (read about stdcall in MinGW, calling conventions).