 Research
 Open Access
 Published:
A parameteradaptive iterative regularization model for image denoising
EURASIP Journal on Advances in Signal Processing volume 2012, Article number: 222 (2012)
Abstract
In this article, an iterative regularization model (IRM) with adaptive parameter is addressed. IRM has gained a lot of attentions. But constant scale parameter becomes very sensitive for the fast convergence. It becomes very important to optimize the scale parameter adaptively. Therefore, we introduce a novel IRM with varying scale parameter because of the fact that when the scale parameter is smaller, the number of the iteration will enhance by IRM. A method to estimate a scale parameter is proposed according to the trend of the scale parameter. And the theoretical justification for this approach can be inferred. Numerical experiments show that the proposed methods with varying scale parameter can efficiently remove noise, reduce the number of iteration, and well preserve the details of images.
Introduction
During the last decade, in spite of the sophistication of the recently proposed methods, some algorithms have not yet attained a desirable level of applicability for image denoising, which is still a challenge at the crossing of functional analysis and statistics. The relations between variational regularization method and wavelet shrinkage have become one of the most active areas of research [1–5].
In this article, we are motivated by the following classical denoising problem of image degraded by additive white Gaussian noise. Given a noisy image f (x, y): Ω→, where Ω is a bounded open subset of σ^{2}, we want to obtain a decomposition equation:
where g(x,y) is the true image and n(x,y) is the noise with (x, y) ∈ Ω and n (x, y) (0, σ^{2})
The most classical variational model is
or its corresponding constrained version
For some scale parameter λ > 0, where BV(Ω) denotes the space of functions with bounded variation on Ω, ·_{2} is L_{2} norm. J(u) is the regularization item and ∥f  u∥_{2}^{2} is the fitting item. λ is chosen to balance inconsistency (first term) and the deviation (second term) from the noise image f(x y) and depends on the noise norm σ. Therefore, a mass of researchers are concentrated on the regularization item J(u). The total variation model of Rudin–Osher–Fatemi (ROF) for image denoising is considered to be the better denoising model. But, there were two serious issues about the ROF model [6–11]. First, it was very complicated to compute the solutions of the optimization problems induced by the variational method. Second, it was difficult to extract textures from images by using the ROF model. For the first issue, Goldstein and Osher recently introduced the split Bregman method for L 1 regularized problems. The Bregman method gave rise to very efficient algorithms for solutions of the ROF model. Meyer [12] did some very interesting analysis by characterizing textures which he defines as “highly oscillatory patterns in image processing” as elements of the dual space of BV(Ω). An iterative regularization model (IRM) [13], which replaces the regularization term by a generalized Bregman distance [14, 15], was proposed. This model is formulated as
Large λ corresponds to very little noise removal, and hence u(x y) is quickly close to f(x y) and the quality of image denoising is not effective. Small λ yields an oversmoothed u(x y) and the iterated times will be enhanced. In spite of the sophistication of the recently proposed methods, most algorithms have not yet attained a desirable level of applicability [15–18].
In this article, we proposed a new denoising method with varying scale parameter where the regularization item is $J\left(u\right)=\underset{\mathrm{\Omega}}{{\displaystyle \iint}}\left\nabla u\rightdxdy$. We deduce a method to gain the scale parameter from the iterative regularization. Finally, some numerical examples are presented and show that our method improves the quality of the image denoising and reduces the optimal number of iterations.
The remainder of this article is organized as follows. In “IRM” section, we mainly review IRM and its some attributes. The proposed method is introduced in “IRM with varying scale parameter” section; the experimental results of our method are given in “Result and discussion” section. This article is summarized in “Conclusion” section.
IRM
IRM makes use of some signals in the removed residual part for these denoising algorithms [19, 20]. For p ∈ ∂J (v), we define the nonnegative quantity
Then, the equivalent representation of Equation (3) is
where u_{0} = 0 and ${D}_{J}^{{p}_{k}}\left(u,{u}_{k}\right)$ are the Bregman distance between u and u_{ k }. As the optimal number of iteration k increases, u is close to the noisy image f. The scale parameter λ tunes the weight between the regularization and fidelity terms. The iterated refinement method yields a welldefined sequence of minimizers {u_{ k }} which satisfies ∥u_{ k } − f∥_{2}^{2} ≤ u_{k − 1} − f_{2}^{2} and if f ∈ BV (Ω), then ${u}_{k}{f}_{2}^{2}\le \frac{J\left(f\right)}{k}$, i.e., u_{ k } converges monotonically to f in L^{2}(Ω) with a rate of $\frac{1}{\sqrt{k}}$. For g ∈ BV (Ω) and γ > 1, we have D(g, u_{ k }) ≤ D(g, u_{k−1}) subject to u_{ k } − f_{2} ≥ γ∥g − f_{2.}
Thus, the distance between a restored image u_{ k } and a possible exact image g is decreasing until the L^{2}– distance of f and u_{ k } is larger than the L^{2}– distance of f and g. This result can be used to construct a stopping rule for our iterative procedure [13].
It should be stressed that the Bregmanbased methodology, in the last few years, has made rapid development due to the tireless efforts of Osher and collaborators [18, 21–23]. A key breakthrough among is that, with adequate initializations, the Bregman method equals to the augmented Lagrangian algorithm [7, 22]. Furthermore, many efficient algorithms are proposed to enable fast implementation [21, 24, 25].
IRM with varying scale parameter
We know that for IRM the bigger the scale parameter λ is, the smaller the number of iteration is to the stop criterion, but u is quickly close to the noise image f, the quality of the image denoising is not ideal. When the scale parameter λ is smaller, the number of the iteration will enhance. Therefore, it is important to choose an optimal value λ.
Varying scale parameter
For that, let $J\left(u\right)=\underset{\mathrm{\Omega}}{{\displaystyle \iint}}\left\nabla u\rightdxdy$, differentiating both sides with respect to u for Equation (3a) we have
Multiplying Equation (6) by $\nabla \xb7\left(\frac{1}{\left\nabla u\right}\nabla u\right)$ and integrating over x and y, we get
Then, we have the following equation
In numerical implementation, we use accordinglyλ_{k+1} denotes λ in Equation (8). Applying the proposed scale parameter to IRM with initial values u^{0} = 0, v^{0} = 0, we obtain different scale parameters λ_{k+1} for different iterations. Equation (3) should be written as
This gives us an adaptive value λ_{k+1}, which appears to converge as k → ∞. The theoretical justification for this approach comes from Appendices 1 and 2.
Initial scale parameter
By the numerical experiment, we discover that the quality of image denoising is not ideal when initial values u_{0} = 0, v_{0} = 0. For example, if the initial condition holds, there is a question that Equation (9a) will be divided by zero.
If we randomly give an initial scale parameter value λ_{0}, we calculate λ_{ k } by
after the iterations are taken some steps. We gain the sequence vector {λ_{ k }} and find that λ_{ k } has some properties as follows:

(a)
the sequence vector {λ _{ k }} is monotonically decreasing as the number of iteration k increases (see Figure 1c);

(b)
as the number of iteration k increases, the sequence vector {λ _{ k }} will at first decrease, and then increase closely to λ _{0} (see Figure 1b);

(c)
the sequence vector {λ _{ k }} is monotonically increasing as the number of iteration k increases (see Figure 1a).
Therefore, we can obtain the initial value of varying scale parameter by the trend of the sequence vector {λ_{ k }} as follows:

(1)
If the sequence vector {λ _{ k }} is monotonically decreasing at first as the number of iteration k increases, we consider that the random selected λ _{0} is contented with the property of (a) or (b). Then, the initial scale parameter λ _{1} of our proposed method is equal to $\stackrel{}{{\text{\lambda}}_{\text{k}}}$. Usually, k is equal to 3.

(2)
If the sequence vector {λ _{ k }} is monotonically increasing as the number of iteration k increases, the random selected λ _{0} is contented with the property of (c). Then, the initial scale parameter of our proposed method ${\text{\lambda}}_{1}=\stackrel{}{{\text{\lambda}}_{1}}$ or λ_{1} = λ^{0}/p with the constant p > 1. Usually, p = 2.
In Figure 1, as the example of ‘Barbara’ image, the trends of the sequence vector {λ_{ k }} are gained when the scale parameter λ_{0} is 8.33, 4.34, and 0.013, respectively.
IRM framework with varying scale parameter
According to the above two sections, our general iterative regularization procedure can be formulated as follows.
Step 1: We randomly select λ_{0}. Let u_{0} = 0, v_{0} = 0 and j = 0, 1, 2…

(1)
According to Equations (11) and (10), we calculate u_{j+1}, v_{j+1}, and λ _{ j } by the number of iteration j. Generally j = 2.

(2)
We observe the trend of the sequence vector {λ _{ k }}. According to the properties of “Initial scale parameter” section, we get the initial value λ _{1} of our proposed method.
Step 2: Let u_{0} = 0, v_{0} = 0 and k = 1, 2…

(1)
According to Equation (9) and the initial value λ _{1}, we calculate u_{k+1}, v_{k+1}, and λ _{k+1}.

(2)
We get image u _{ k } and stop the iteration when ∥f  u∥_{ k } ≤ σ (as the stopping criterion).
Result and discussion
All solutions to the variational problem were obtained using gradient descent in a standard fashion [21–28]. Now, we use Chambolle Algorithm [8]. The only nontrivial difficulty comes when ∇u ≈ 0. We fix this, as is usual, by perturbing $J\left(u\right)=\underset{\mathrm{\Omega}}{{\displaystyle \iint}}\left\nabla u\rightdxdy$ to ${J}_{\epsilon}\left(u\right)=\underset{\mathrm{\Omega}}{{\displaystyle \iint}}\sqrt{{\left\nabla u\right}^{2}+{\epsilon}^{2}}dxdy$, where ε is a small positive number. To be extent, the ‘staircasing’ effect of this method can be decreased. In our calculations, we too k = 10^{−12}; the step of iteration unit τ for Chambolle Algorithm is 0.2. Without loss of generality, the performance of the denoising algorithms is measured in terms of peak signaltonoiseratio (PSNR) [29], which can be defined as follows
where f is the original image and u is the denoising image.
Convergence analysis
Figures 2 and 3 show the results with constant and varying scale parameter of IRM with ‘cameramen’ image added Gauss noise σ = 20 when λ is smaller and bigger, respectively. In Figure 2, the first row results show that more iteration steps are required to stop criterion with smaller scale parameter λ_{0} = 0.67; the second row results show that our proposed methods require less iterations to get the optimal denoising results. At first, we used constant scale parameter λ_{0} = 0.67 to iterate three times and got a sequence vector {λ_{ k }} decreased in the first image of the third row. According to Equation (11), we got the initial value λ_{1} = λ_{2} = 5.74. The last two plots (i) and (j) show that ∥f − u∥_{k 2} decreases monotonically with the iteration, first dropping below σ at the optimal iterate k = 12 and 2, respectively. It shows that our proposed method converge faster than IRM with the constant scale parameter. In Figure 3, as can be seen, with large scale parameter λ_{0} = 10 the original IRM convergences to the noisy image f quickly, and only one iterative needed to reach the stop criterion. Obviously, the denoising result is not satisfied. However, promising result is obtained by our varying scale parameter strategy where the initial value λ_{1} = λ_{2} = 5.74 according to Equation (10).
Preserved textures analysis
Figure 4 shows the denoising results of ‘Barbara’ image with Gauss white noise σ = 25.5. The constant scale parameter λ for IRM is 1. Compared with the constant scale parameter for IRM in Figure 4c–f, our proposed method can preserved more textures in Figure 4g–j. The last two plots (k) and (l) show that ∥f − u∥_{k 2} decreases monotonically with the iterations, first dropping below σ at the optimal iterate k = 12 and 2, respectively. It shows that our proposed method converges faster than IRM with the constant scale parameter.
Denoising analysis for MRI coronal brain
The denoising results of MRI coronal brain image with Gauss white noise σ = 53.83 are shown in Figure 5. The constant scale parameter λ for IRM is 1. Compared with the constant scale parameter for IRM in Figure 5c–f, our proposed method can preserve more textures in Figure 5g–j. The last two plots (k) and (l) show that ∥f − u∥_{k 2} decreases monotonically with the iterations, first dropping below σ at the optimal iterate k = 13 and 3, respectively. It shows that our proposed method converges faster than IRM with the constant scale parameter and has more texture details in the denoised image.
Computational cost analysis
We have made a comparison in terms of computational time (see Tables 1 and 2) by MATLAB 7.1, which is used on a PC equipped with AMD 2.31 GHz CPU and 3 GB RAM. In fact, it should be noted that the term “Fast” is relatively for computing (9a) when compared with computing the subproblem (9b), i.e., in this article even we use the efficient Chambolle Algorithm to solve the subproblem (9b) and set the inner iterations is 40. To take the results of “Preserved textures analysis” section for denoising ‘Barbara’ image (256 × 256) as an example, the average time of computing (9b) once is 1.281 s, while that of (9a) once is only 0.026 s. Moreover, combined with fact that the outer iterations of the conventional IRM is 13 while our adaptive IRM is 3, the whole computational time of the conventional IRM is 13.708 s while that of our method is 3.136 s. Therefore, we know that our adaptive scheme is really faster than the conventional one.
In addition, we have made a comparison between wavelet + wiener, curvelets include hard threshold, soft threshold, and block threshold regulation algorithm and our method. Denoising results of Lena image are shown that our algorithm improves PSNR than traditional method in Table 3.
Conclusion
A novel IRM with adaptive scale parameter is proposed in order to decrease the sensitivity of constant scale parameter, optimize the scale parameter adaptively in the IRM, and attain a desirable level of applicability for image denoising. We replace the classic regularization item and deduce the equation of the adaptive scale parameter, because we know that the scale parameter is smaller, the number of the iteration will enhance by IRM. Then, the rule of varying scale parameter by the trend of the sequence vector is attained. A new iterative scale parameter λ is obtained according to the trend of the sequence vector. In general, we can get the initial scale parameter λ just using three steps of iteration. We have seen with practical examples that our proposed method can reduce the number of iterations. Thus, a fast and robust method is got.
Appendix 1
A constrained problem is defined as
We know that the constraint equation is N^{T}X − b = 0. The basic assumption is that X lies in the subspace tangent to the active constraints, i.e., X_{i+1} = X_{ i } + αS, where S is the direction with the most negative directional derivative and α is the iterative step length, both X_{ i } and X_{i+1} satisfy the constraint equations. Therefore, we obtain
If we want the steepest descent direction satisfying Equation (14), we can pose the problem as
The Euler–Lagrange equation (15) has the formulation
The derivative of L with respect to S is
Recall that N^{T}S = 0 in Equation (14) and multiplying Equation (17) by N^{T}, we get
Therefore, we get the value
So, the proposition holds.
Appendix 2
In this article, we want to solve the question
It is equivalent to
where ${\lambda}_{1}=\frac{1}{\lambda}$. Then, Equation (21) is rewritten to
Since $\underset{\mathrm{\Omega}}{{\displaystyle \iint}}\left\nabla u\rightdxdy=\nabla {u}_{1}\approx \sqrt{\nabla {u}_{1}^{2}+\epsilon}=<\frac{{\nabla}^{*}\nabla u}{\sqrt{\nabla {u}_{1}^{2}+\epsilon}},u>$, we let $\frac{{\nabla}^{*}\nabla u}{\sqrt{\nabla {u}_{1}^{2}+\epsilon}}$ be approximated by $\frac{{\nabla}^{*}\nabla {u}_{k}}{\sqrt{\nabla {{u}_{k}}_{1}^{2}+\epsilon}}$, and let $u=X,b=0,N=\frac{{\nabla}^{*}\nabla {u}_{k}}{\sqrt{\nabla {{u}_{k}}_{1}^{2}+\epsilon}}$ and $\left(u\right)=\frac{1}{2}f+{v}_{k}{u}_{2}^{2}$, we have $\underset{\mathrm{\Omega}}{{\displaystyle \iint}}\left\nabla u\rightdxdy=={N}^{T}u\ge 0$, then according to Appendix 1, we obtain
Therefore, we obtain the parameter
So, the proposition holds.
References
 1.
Rudin L, Osher S, Fatemi E: Nonlinear total variation based noise removal algorithms. Physica D 1992, 60: 259268. 10.1016/01672789(92)90242F
 2.
Devore R, Jawerth B, Lucier B: Image compression through wavelet transform coding. IEEE Trans. Inf. Theory 1992, 38: 719746. 10.1109/18.119733
 3.
Donoho DL: Denoising by softthreshold. IEEE Trans. Inf. Theory 1995, 41(3):613627. 10.1109/18.382009
 4.
Dabov K, Foi A, Katkovnik V, Egiazarian K: Image denoising by sparse 3D transformdomain collaborative filtering. IEEE Trans. Image Process. 2007, 16: 20802095.
 5.
Yan R, Shao L, Cvetkovic SD, Klijn J: Improved nonlocal means based on preclassification and invariant block matching. IEEE/OSA J. Disp. Technol. 2012, 8(4):212218.
 6.
Osher S, Solè A, Vese L: Image decomposition and restoration using total variation minimization and the H1 norm. SIAM J. Multiscale Model. Simulat. 2003, 1(3):349370. 10.1137/S1540345902416247
 7.
Burger M, Gilboa G, Osher S, Xu JJ: Nonlinear inverse scale space methods. Commun. Math. Sci. 2006, 4(1):175208.
 8.
Chambolle A, DeVore R, Lee NY, Lucier B: Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage. IEEE Trans. Image Process. 1998, 7(3):319335.
 9.
Chambolle A, Lucier BJ: Interpreting translationinvariant wavelet shrinkage as a new image smoothing scale space. IEEE Trans. Image Process. 2001, 10: 9931000. 10.1109/83.931093
 10.
Steidl G, Weickert J, Brox T, MraZek P, Welk M: On the equivalence of soft wavelet shrinkage, total variation diffusion, total variation regularization, and SIDEs. SIAM J. Numer. Anal. 2004, 42(2):686713. 10.1137/S0036142903422429
 11.
Xu J, Osher S: Iterative regularization and nonlinear inverse scale space applied to waveletbased denoising. IEEE Trans. Image Process. 2007, 16(2):534544.
 12.
Meyer Y: Oscillating Patterns in Image Processing and Nonlinear Evolution Equations. American Mathematical Society, Boston; 2002.
 13.
Osher S, Burger M, Goldfarb D, Xu J, Yin W: An iterative regularization method for total variation based image restoration. Multiscale Model. Simulat. 2005, 4: 460489. 10.1137/040605412
 14.
Bregman L: The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming. U.S.S.R. Comput. Math. Math. Phys. 1967, 7: 200217.
 15.
Hao BB, Li M, Feng XC: Wavelet iterative regularization for image restoration with varying scale parameter. Signal Process. Image Commun. 2008, 23(6):433441. 10.1016/j.image.2008.04.006
 16.
Liu B, King K, Steckner M, Xie J, Sheng J, Ying L: Regularized sensitivity encoding (SENSE) reconstruction using Bregman iterations. Magn. Reson. Med. 2009, 61(1):145152. 10.1002/mrm.21799
 17.
Chambolle A: An algorithm for total variation minimization and applications. J. Math. Imag. Vis. 2004, 20: 8997.
 18.
Vese L, Osher S: Modeling textures with total variation minimization and oscillating patterns in image processing. J. Sci. Comput. 2003, 19: 553572. 10.1023/A:1025384832106
 19.
Weissman T, Ordentlich E, Seroussi G, Verdu S, Weinberger M: Universal discrete denoising: known channel. IEEE Trans. Inf. Theory 2005, 51: 528.
 20.
Perona P, Malik J: Scale space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 1990, 12: 629639. 10.1109/34.56205
 21.
Goldstein T, Osher S: The split Bregman method for L1regularized problems. SIAM J. Imag. Sci. 2009, 2(2):323343. 10.1137/080725891
 22.
Yin W, Osher S, Goldfarb D, Darbon J: Bregman iterative algorithms for l1minimization with applications to compressed sensing. SIAM J. Imag. Sci. 2009, 1: 142168.
 23.
He L, Chang TC, Osher S, Fang T, Speier P: MR image reconstruction by using the iterative refinement method and nonlinear inverse scale space methods. 2006. Technical Report CAM Report 06–35, UCLA
 24.
Esser E: Applications of Lagrangianbased alternating direction methods and connections to split Bregman. 2009. CAM Report 09–31, UCLA
 25.
Wang Y, Yang J, Yin W, Zhang Y: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imag. Sci. 2008, 1(3):248272. 10.1137/080724265
 26.
Ma J: Compressed sensing by inverse scale space and curvelet thresholding. Appl. Math. Comput. 2008, 206(2):980988. 10.1016/j.amc.2008.10.011
 27.
Ma J, Le DF: Deblurring from highly incomplete measurements for remote sensing. IEEE Trans. Geosci. Remote Sensing (GeoRS) 2009, 3(47):792802.
 28.
Gilboa G, Sochen N, Zeevi YY: Texture preserving variational denoising using an adaptive fidelity term. Proc VLSM Nice, France 1 2003, 137144.
 29.
Shao L, Zhang H, de Haan G: An overview and performance evaluation of classificationbased least squares trained filters. IEEE Trans. Image Process. 2008, 17: 1772178.
Acknowledgment
This study was supported by the National Natural Science Foundation of China under the Grant nos. 60702069, 30300443 and 61105035; the Research Project of Department of Education of Zhejiang Province, China under the Grant no. 20060601; The Science Foundation of Zhejiang SciTech University of China under the Grant no. 0604039Y; the Natural Science Foundation of Zhejiang Province of China under the Grant no. Y1080851 and Y12H290045; the Research Project of 2011 overseas students of Zhejiang Province of China under the Grant no. 1104707M; Qianjiang talents project of Science and Technology Department of Zhejiang province of China under the grant no. 2012R10054.
Author information
Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Li, W., Zhao, C., Liu, Q. et al. A parameteradaptive iterative regularization model for image denoising. EURASIP J. Adv. Signal Process. 2012, 222 (2012). https://doi.org/10.1186/168761802012222
Received:
Accepted:
Published:
Keywords
 Iterative regularization
 Total variation
 Variational methods
 Image denoising