The color image has a various edge profile. In the conventional single kernel filter has the drawbacks of exhausting background noises or insensitivity to the dull edges. In the previous paper, we proposed a new image sharpening method adaptive to the edge profiles. In this paper, we present its advanced model, which has both sharpening and smoothing functions. In addition, the paper assesses the edge sharpness factors by introducing the indices, such as ES (Edge Sharpness), FS (Spatial Frequency Sharpness), and Nf (Flat Area Noise). The improved model makes the flat area noises intentionally smoothed with preserving the enhanced edges. The sharpening filter is applied only to the luminance Y image to keep the gray balance. After pre-scanning the Y image with sharp GD filter, the edge map is generated by classifying the edge types into hard, medium, soft, and flat zones. The multiple GD operators with different deviations, σ₁, σ₂, and σ₃ are selectively applied to the corresponding edge zones of Y image by looking up the edge map. Here the smoothing filters are applied only to the flat zones to reduce the background noises. In simple, the normal Gaussian filter is used as a noise smoother. In comparison with conventional method, the proposed model worked excellent to sharpen the different edge slopes naturally together with dramatically reducing the background noises.