Simulation of fluid-structure interaction (FSI) of flexible bodies are challenging due to complex geometries and freely moving boundaries. Immersed boundary method has found to be an efficient technique for dealing with FSI problems because of the use of non-body-fitted mesh and simple implementation. In the present work, we developed a FSI solver by coupling a direct forcing immersed boundary method for the fluid with a finite difference method of the structure. Several flow problems are simulated to validate our method. The testing cases include flow over a stationary cylinder and flat plate, two-dimensional flow past an inextensible flexible filament and three-dimensional flow past a flag. The results obtained agree well with those from previously published literatures.