This paper presents a system framework to realize the automatic visual inspection with random initial object poses. A series of Computer Graphics (CG) images rendered in OpenGL are compared with a real scene image from a fixed environment camera. With an iterative heuristic search approach for the maximum similarity in these comparisons, robust 3D object pose estimation has been obtained to place an inspection camera to its new initial pose. Then a visual servo technique is used to accurately place this camera so that a reliable visual inspection can be carried out. Contrast with those feature-based or texture-based approaches where textures are required, only the object's geometry information is needed in this framework, therefore this system can deal with the untextured objects (for instance metal parts) well. This paper describes the system configuration and operational procedure in detail. Experiments results are presented to evaluate the system performance.