Probabilistic Approximation Algorithm for Set Cover
1 The Weighted Set Cover Problem
In the Weighted Set Cover (WSC) problem, the input is a triple
where
is a universe of
elements,
is a family of
sets such that each
and
, and
is a weight function that assigns positive weight to each set in
. A solution
for the WSC problem is valid if and only if
, the weight of the solution is
. Note that there is always a valid solution, and the goal in the WSC problem is to find such one with minimum weight.
Weighted Set Cover is “hard” in many aspects. First, it is hard to efficiently find the exact solution, that is, the problem is NP-Complete so no time-efficient algorithm is known for it. Moreover, it is also hard to approximate the solution better than a
factor. Formally, we said that a solution
is a
approximation for WSC if
, when
is an optimal solution with minimum weight. The WSC is hard to approximate in the sense that any
approximation for that problem is NP-Complete by itself, and that is for any
.
In this post we will describe a probabilistic
approximation algorithm for WSC which runs in
time, and
is some positive constant that controls the success probability of the algorithm.
2 Equivalent Formulation
We can reformulate the Weighted Set Cover problem as a system of linear equations as follows: For any set
define an indicator variable
and let
, thus if
then
, and
otherwise. The objective is then to minimize
subject to the
following linear constrains
Given a solution
to that system of equations, we can define the corresponding solution for WSC as
. Note that by this construction
must be valid, moreover, the weights are the same:
, so if
is minimal then also
. The proof of the other direction (from WSC to the system of equations) is similar.
Because the linear-time mapping from any optimal solution of one problem to an optimal solution for the other, we can conclude that solving this system of equations is NP-Hard. However, if we relax the “integer requirement”, i.e. that
, then the problem becomes much easier. That is, when
for any
, that system becomes a convex optimization problem, which we can solved with a polynomial-time algorithm. From now on, we will refer only to the convex version, and denote its (exact) solution as fractional solution.
So suppose we have a fractional optimal solution for the linear system
. Note that
for any optimal solution
for WSC, because the convex formulation allows for more possible valid solutions so the minimum weight can only be smaller than of
.
How do we get from
a solution for the WSC? The nice trick here is probabilistic rounding. That is, we will treat each
as the probability for selecting
.
We define the following procedure
as an helper function:
-
Repeat for
times:
- Initialize .
- Solve the convex optimization version of the WSC to get an optimal fractional solution .
- For each , add it to with probability .
- Let the “solution” for the current iteration be .
- Output
The final algorithm is just applying the
procedure
independent times, and keeping the candidate solutions
, then returning the
solution with minimum weight among those candidates.
Note that all the procedures above have polynomial time (in
and
), therefore the total running time of the algorithm is
.
3 The Solution is Approximately Optimal
We want to show that with high probability
. It is easy to see that for each iteration of the sub-procedure
, the intermediate set
has expected weight equals to the weight of the optimal fractional solution. Formally,
note that the optimal value of all the fractional solutions is the same so we will remove the subscript
. Next, we continue to calculate the expected weight of the output solution of the sub-procedure, that is, the weight of the candidate
It left to show that with high probability all the candidates
are within the weight bound of
, and because
is being selected from those candidates, we conclude that it also has
. In order to get high probability for a single candidate, we can use Markov’s inequality as follows. Define a “failure” of a candidate
as
, then the probability of a single candidate to fail is bounded by
So with probability of at least
the candidate has
as required.
When considering the final solution
, it fails only when all the candidates fail and the probability for that is
. So we have high constant probability to get a solution that is
approximation for the optimal solution.
4 The Solution is Probably Valid
What is the probability that the returned solution
is not valid? that is, there is at least one element
that is not covered by
. We analyze that by first considering an arbitrary element
and an intermediate solution
of the sub procedure
. Suppose that
belongs to
sets, and for simplicity suppose they are
. We know that their corresponding variables have
by the requirements of the linear system. The probability to leave all these sets outside
is
that is, the probability that an intermediate solution
will not cover
is up to
.
Next, we calculate the probability that
is not covered by a
where
for some selected candidate
. Then it must be that
is not covered by any intermediate solution
of the procedure
, and that will happen with probability
. Applying the union-bound on all the
elements, we conclude that with probability of at most
, the selected candidate
will not cover the entire universe. In other words, with at least
probability,
is a valid solution.