A so-called test installation is a local installation of SMF or any other software package, right on your own computer. With it you can test modifications done to the forum before applying them on your production site.
Software you need
Depending on your platform, you need to install a software package.
WebMatrix
Microsoft has made an IDE available for Windows with which you can install various software installations with a few clicks.
AMP packages
A list of all AMP (Apache-MySQL-PHP) packages, required to run SMF and other softwares, can be found here: Wikipedia: List of AMP packages
Some of these packages include phpMyAdmin or a similar tool for database management which you may need. If not, you can install phpMyAdmin yourself from here, or a smaller alternative, Adminer, from here.
Installing SMF
After you have installed the program appropriate for your system, you can proceed and install SMF on your computer.