The color image has various edge profiles. In the conventional single kernel sharpness filter has the drawbacks of exhausting background noises or insensitivity to the dull edges. In the previous paper, we proposed an adaptive image sharpening method to the edge profiles. This paper, reports its advanced model, which has both sharpening and smoothing functions. In addition, the paper assesses the edge sharpness factors by introducing the indices, such as ES (Edge Sharpness), FS (Spatial Frequency Sharpness), and Nf (Flat Area Noise). The improved model makes the flat area noises intentionally smoothed with preserving the enhanced edges. A prescanning spatial filter generates the edge map to classify the edge types into hard, medium, soft, and flat zones. The multiple GD (Gaussian Derivative) operators with different deviations are selectively applied to the corresponding edge zones by looking up the edge map. Here the smoothing filters are applied only to the flat zones to reduce the background noises. In comparison with the single kernel filter method, the proposed model worked excellent to sharpen the different edge slopes naturally together with dramatically reducing the background noises.