Towards NearlyLinear Time Algorithms for Submodular Maximization with a Matroid Constraint
I C A L P
Towards NearlyLinear Time Algorithms for Submodular Maximization with a Matroid Constraint
Huy L. Nguyen 0 1 2
Category Track A: Algorithms, Complexity and Games
0 Alina Ene Department of Computer Science, Boston University , MA , USA
1 Funding Alina Ene: Partially supported by NSF CAREER grant CCF1750333 and NSF grant CCF1718342. Huy L. Nguyen: Partially supported by NSF CAREER grant CCF1750716
2 College of Computer and Information Science, Northeastern University , Boston, MA , USA
We consider fast algorithms for monotone submodular maximization subject to a matroid constraint. We assume that the matroid is given as input in an explicit form, and the goal is to obtain the best possible running times for important matroids. We develop a new algorithm for a general matroid constraint with a 1 ? 1/e ? approximation that achieves a fast running time provided we have a fast data structure for maintaining an approximately maximum weight base in the matroid through a sequence of decrease weight operations. We construct such data structures for graphic matroids and partition matroids, and we obtain the first algorithms for these classes of matroids that achieve a nearlyoptimal, 1 ? 1/e ? approximation, using a nearlylinear number of function evaluations and arithmetic operations. 2012 ACM Subject Classification Theory of computation ? Submodular optimization and polymatroids
and phrases submodular maximization; matroid constraints; fast running times

Acknowledgements This work was done in part while the authors were visiting the Simons Institute
for the Theory of Computing.
In this paper, we consider fast algorithms for monotone submodular maximization subject
to a matroid constraint. Submodular maximization is a central problem in combinatorial
optimization that captures several problems of interest, such as maximum coverage, facility
location, and welfare maximization. The study of this problem dates back to the seminal work
of Nemhauser, Wolsey and Fisher from the 1970?s [20, 21, 12]. Nemhauser et al. introduced a
very natural Greedy algorithm for the problem that iteratively builds a solution by selecting
the item with the largest marginal gain on top of previously selected items, and they showed
that this algorithm achieves a 1 ? 1/e approximation for a cardinality constraint and a 1/2
approximation for a general matroid constraint. The maximum coverage problem is a special
case of monotone submodular maximization with a cardinality constraint and it is 1 ? 1/e
ET
A
CS
hard to approximate [10], and thus the former result is optimal. Therefore the main question
that was left open by the work of Nemhauser et al. is whether one can obtain an optimal,
1 ? 1/e approximation, for a general matroid constraint.
In a celebrated line of work [6, 24], Calinescu et al. developed a framework based on
continuous optimization and rounding that led to an optimal 1 ? 1/e approximation for
the problem. The approach is to turn the discrete optimization problem of maximizing
a submodular function f subject to a matroid constraint into a continuous optimization
problem where the goal is to maximize the multilinear extension F of f (a continuous
function that extends f ) subject to the matroid polytope (a convex polytope whose vertices
are the feasible integral solutions). The continuous optimization problem can be solved
approximately within a 1 ? 1/e factor using a continuous Greedy algorithm [24] and the
resulting fractional solution can be rounded to an integral solution without any loss [1, 6, 8].
The resulting algorithm achieves the optimal 1 ? 1/e approximation in polynomial time.
Unfortunately, a significant drawback of this approach is that it leads to very high running
times. Obtaining fast running times is a fundamental direction both in theory and in practice,
due to the numerous applications of submodular maximization in machine learning, data
mining, and economics [18, 17, 14, 16, 9]. This direction has received considerable attention [2,
11, 3, 19, 5, 7], but it remains a significant challenge for almost all matroid constraints.
Before discussing these challenges, let us first address the important questions on how
the input is represented and how we measure running time. The algorithms in this paper as
well as prior work assume that the submodular function is represented as a value oracle that
takes as input a set S and returns f (S). For all these algorithms, the number of calls to the
value oracle for f dominates the running time of the algorithm (up to a logarithmic factor),
and thus we assume for simplicity that each call takes constant time.
The algorithms fall into two categories with respect to how the matroid is represented:
the independence oracle algorithms assume that the matroid is represented using an oracle
that takes as input a set S and returns whether S is feasible (independent); the representable
matroid algorithms assume that the matroid is given as input in an explicit form. The
representable matroid algorithms can be used for only a subclass of matroids, namely those
that can be represented as a linear matroid over vectors in some field1, but this class includes
practicallyrelevant matroids such as the uniform, partition, laminar, graphical, and general
linear matroids. The oracle algorithms apply to all matroids, but they are unlikely to lead to
the fastest possible running times: even an ideal algorithm that makes only O(k) independence
calls has a running time that is ?(k2) in the independence oracle model (each oracle call
needs to read its input, which takes ?(k) time in the worst case), even if the matroid is a
representable matroid such as a partition or a graphic matroid. This is because Thus there
have always been parallel lines of research for representable matroids and general matroids.
This work falls in the first category, i.e., we assume that the matroid is given as input
in an explicit form, and the goal is to obtain the best possible running times. Note that,
although all the representable matroids are linear matroids, it is necessary to consider each
class separately, since they have very different running times to even verify if a given solution
is feasible: for simple explicit matroids such as a partition or a graphic matroid, checking
whether a solution is feasible takes O(n) time, where n is the size of the ground set of
the matroid; for general explicit matroids represented using vectors in some field, checking
whether a solution is feasible takes O(k?) time, where k is the rank of the matroid and ? is
the exponent for fast matrix multiplication.
1 In a linear matroid, the ground set is a collection of n vectors and a subset of the vectors is feasible
(independent) if the vectors are linearly independent.
Since in many practical settings only nearlylinear running times are feasible, an important
question to address is:
For which matroid constraints can we obtain
a nearlyoptimal 1 ? 1/e ? approximation in nearlylinear time?
Prior to this work, the only example of such a constraint was a cardinality constraint. For
a partition matroid constraint, the fastest running time is ?(n3/2) in the worst case when
k = ?(n) [5]. For a graphical matroid constraint, no faster algorithms are known than
a general matroid, and the running time is ?(n2) in the worst case when k = ?(n) [3].
Obtaining a bestpossible, nearlylinear running time has been very challenging even for
these classes of matroids for the following reasons:
The continuous optimization is a significant time bottleneck. The continuous
optimization problem of maximizing the multilinear extension subject to the matroid polytope is an
integral component in all algorithms that achieve a nearlyoptimal approximation guarantee.
However, the multilinear extension is expensive to evaluate even approximately. To achieve
the nearlyoptimal approximation guarantees, the evaluation error needs to be very small
and in a lot of cases, the error needs to be O(n?1) times the function value. As a result, a
single evaluation of the multilinear extension requires ?(n) evaluations of f . Thus, even a
very efficient algorithm with O(n) queries to the multilinear extension would require ?(n2)
running time.
Rounding the fractional solution is a significant time bottleneck as well. Consider a
matroid constraint of rank k. The fastest known rounding algorithm is the swap rounding,
which requires k swap operations: in each operation, the algorithm has two bases B1 and B2
and needs to find x ? B1, y ? B2 such that B1 \ {x} ? {y} and B2 \ {y} ? {x} are bases. The
typical implementation is to pick some x ? B1 and try all y in B2, which requires us to check
independence for k solutions. Thus, overall, the rounding algorithm checks independence for
?(k2) solutions. Furthermore, each feasibility check takes ?(k) time just to read the input.
Thus a generic rounding for a matroid takes ?(k3) time.
Thus, in order to achieve a fast overall running time, one needs fast algorithms for both
the continuous optimization and the rounding. In this work, we provide such algorithms
for partition and graphic matroids, and we obtain the first algorithms with nearlylinear
running times. At the heart of our approach is a general, nearlylinear time reduction that
reduces the submodular maximization problem to two data structure problems: maintain an
approximately maximum weight base in the matroid through a sequence of decreaseweight
operations, and maintain an independent set in the matroid that allows us to check whether
an element can be feasibly added. This reduction applies to any representable matroid, and
thus it opens the possibility of obtaining faster running times for other classes of matroids.
1.1
Our contributions
We now give a more precise description of our contributions. We develop a new algorithm
for maximizing the multilinear extension subject to a general matroid constraint with a
1 ? 1/e ? approximation that achieves a fast running time provided we have fast data
structures with the following functionality:
A maximum weight base data structure: each element has a weight, and the goal is to
maintain an approximately maximum weight base in the matroid through a sequence of
operations, where each operation can only decrease the weight of a single element;
An independent set data structure that maintains an independent set in the matroid and
supports two operations: add an element to the independent set, and check whether an
element can be added to the independent set while maintaining independence.
I Theorem 1. Let f be a monotone submodular function and let M be a matroid on a ground
set of size n. Let F be the multilinear extension of f and P (M) be the matroid polytope
of M. Suppose that we have a data structure for maintaining a maximum weight base and
independent set as described above. There is an algorithm for the problem maxx?P (M) F (x)
that achieves a 1 ? 1/e ? approximation using O(n poly(log n, 1/ )) calls to the value oracle
for f , data structure operations, and additional arithmetic operations.
Using our continuous optimization algorithm and additional results that are included
in the full version of this paper, we obtain the first nearlylinear time algorithms for both
the discrete and continuous problem with a graphic and a partition matroid constraint. In
the graphic matroid case, the maximum weight base data structure is a dynamic maximum
weight spanning tree (MST) data structure and the independent data structure is a dynamic
connectivity data structure (e.g., unionfind), and we can use existing data structures that
guarantee a polylogarithmic amortized time per operation [15, 13, 23]. For a partition
matroid, we provide data structures with a constant amortized time per operation. We
also address the rounding step and provide a nearlylinear time algorithm for rounding a
fractional solution in a graphic matroid. A nearlylinear time rounding algorithm for a
partition matroid was provided in [5].
I Theorem 2. There is an algorithm for maximizing a monotone submodular function subject
to a generalized partition matroid constraint that achieves a 1 ? 1/e ? approximation using
O(n poly(1/ , log n)) function evaluations and arithmetic operations.
I Theorem 3. There is an algorithm for maximizing a monotone submodular function
subject to a graphic matroid constraint that achieves a 1 ? 1/e ? approximation using
O(n poly(1/ , log n)) function evaluations and arithmetic operations.
Previously, the best running time for a partition matroid was ?(n3/2 poly(1/ , log n)) in
the worst case when k = ?(n) [5]. The previous best running time for a graphic matroid is
the same as the general matroid case, which is ?(n2 poly(1/ , log n)) in the worst case when
k = ?(n) [3].
As shown by Vondrak [24], there is an efficient reduction from the submodular welfare
maximization problem to the submodular maximization problem with a partition matroid
constraint. Using this reduction and our algorithm for a partition matroid, we obtain a
nearlylinear time algorithm for welfare maximization as well.
I Theorem 4. There is a 1 ? 1/e ? approximation algorithm for submodular welfare
maximization using O(npoly(1/ , log n)) function evaluations and arithmetic operations.
We conclude with a formal statement of the contributions made in this paper on which
the results above are based.
I Theorem 5. There is a dynamic data structure for maintaining a maximum weight base in
a partition matroid through a sequence of decrease weight operations with an O(1) amortized
time per operation.
I Theorem 6. There is a randomized algorithm based on swap rounding for the graphic
matroid polytope that takes as input a point x represented as a convex combination of bases
and rounds it to an integral solution S such that E[f (S)] ? F (x). The running time of the
algorithm is O(nt log2 n), where t is the number of bases in the convex combination of x.
1.2
Technical overview
The starting point of our approach is the work [5]. They observed that the running time
of the continuous algorithm using the multilinear extension of [3] depends on the value
of the maximum weight base when the value is measured in the modular approximation
f 0(S) = Pe?S f (e). It is clear that this approximation is at least the original function
and it can be much larger. They observed that the running time is proportional to the
ratio between the maximum weight base when weights are measured using the modular
approximation compared with the optimal solution when weights are measured using the
original function. On the other hand, in the greedy algorithm, the gain in every greedy step
is proportional to the maximum weight base when weights are measured using the modular
approximation. Thus, the discrete greedy algorithm makes fast progress precisely when the
continuous algorithm is slow and vice versa. Therefore, one can start with the discrete greedy
algorithm and switch to the continuous algorithm when the maximum weight solution is
small even when weights are measured using the modular approximation.
Our algorithm consists of two key components: (1) a fast dynamic data structure for
maintaining an approximate maximum weight base through a sequence of greedy steps, and
(2) an algorithm that makes only a small number of queries to the data structure. Even if
fast dynamic data structures are available, previous algorithms including that of [5] cannot
achieve a fast time, since they require ?(nk) queries to the data structure: the algorithm of
[5] maintains the marginal gain for every element in the current base and it updates them
after each greedy step; since each greedy step changes the marginal gain of every element in
the base, this approach necessitates ?(k) data structure queries per greedy step.
Our new approach uses random sampling to ensure that the number of queries to the
data structure is nearlylinear. After each greedy step, our algorithm randomly samples
elements from the base to check and update the marginal gains. Because of sampling, it can
only ensure that at least 1/2 of the elements in every value range have good estimates of
their values. However, this is sufficient for maintaining an approximate maximum weight
base. The benefit is that the running time becomes much faster: the number of checks
that do not result in updates is small and if we make sure that an update only happens
when the marginal gain change by a factor 1 ? then the total number of updates is at
most O(n log n/ ). Thus we obtain an algorithm with only a nearlylinear number of data
structure queries and additional running time for any matroid constraint.
Our approach reduces the algorithmic problem to the data structure problem of
maintaining an approximate maximum weight base through a sequence of value updates. In fact,
the updates are only decrement in the values and thus can utilize even the decremental data
structures as opposed to fully dynamic ones. In the case of a partition matroid constraint,
one can develop a simple adhoc solution. In the case of a graphic matroid, one can use
classical data structures for maintaining minimum spanning trees [15].
In both cases, fast rounding algorithms are also needed. The work [5] gives an algorithm
for the partition matroid. We give an algorithm for the graphic matroid based on swap
rounding and classical dynamic graph data structures. To obtain fast running time, in each
rounding step, instead of swapping a generic pair, we choose a pair involving a leaf of the
spanning tree.
1.3
Basic definitions and notation
Submodular functions. Let f : 2V ? R+ be a set function on a finite ground set V of size
n := V . The function is submodular if f (A) + f (B) ? f (A ? B) + f (A ? B) for all subsets
A, B ? V . The function is monotone if f (A) ? f (B) for all subsets A ? B ? V . We assume
that the function f is given as a value oracle that takes as input any set S ? V and returns
f (S). We let F : [0, 1]V ? R+ denote the multilinear extension f . For every x ? [0, 1]V ,
we have
F (x) =
X f (S) Y xe Y (1 ? xe) = E[R(x)],
S?V e?S e?/S
where R(x) is a random set that includes each element e ? V independently with
probability xe.
Matroids. A matroid M = (V, I) on a ground set V is a collection I of subsets of V , called
independent sets, that satisfy certain properties (see e.g., [22], Chapter 39). In this paper, we
consider matroids that are given to the input to the algorithm. Of particular interest are the
partition and graphic matroids. A generalized partition matroid is defined as follows. We are
given a partition V1, V2, . . . , Vh of V into disjoint subsets and budgets k1, k2, . . . , kh. A set S
is independent (S ? I) if S ? Vi ? ki for all i ? [h]. We let k = Ph
i=1 ki denote the rank of
the matroid. A graphic matroid is defined as follows. We are given a connected graph on
k + 1 vertices and n edges. The independent sets of the matroid are the forests of this graph.
Additional notation. Given a set S ? I, we let fS denote the function fS : 2V \S ? R?0
such that fS (S0) = f (S0 ? S) ? f (S) for all S0 ? V \ S. We let M/S = (V \ S, I0) denote the
matroid obtained by contracting S in M, i.e., S0 ? I0 iff S0 ? S ? I. We let P (M) denote
the matroid polytope of M: P (M) is the convex hull of the indicator vectors of the bases of
M, where a base is an independent set of maximum size.
Constant factor approximation to f (OPT). Our algorithm needs a O(1) approximation
to f (OPT). Such an approximation can be computed very efficiently (see e.g. [5], Lemma 3.2).
1.4
Paper organization
In Section 2, we describe our algorithm for the continuous optimization problem of maximizing
the multilinear extension subject to a general matroid constraint, with the properties stated
in Theorem 1. As discussed in the introduction, our algorithm uses certain data structures
to achieve a fast running time. In the full version of the paper, we show how to obtain these
data structures for partition and graphic matroids. By combining these data structures
with the results of Section 2, we obtain nearlylinear time algorithms for the continuous
problem of maximizing the multilinear extension subject to a partition and graphic matroid
constraint. To obtain a fast algorithm for the discrete problem, we also need a fast algorithm
to round the fractional solution. Buchbinder et al. [5] give a nearlylinear time rounding
algorithm for a partition matroid. In the full version of the paper, we give a nearlylinear
time rounding algorithm for a graphic matroid, and prove Theorem 6. These results together
give Theorems 2 and 3.
2
The algorithm for the continuous optimization problem
In this section, we describe and analyze our algorithm for the problem maxx?P (M) F (x) for
a general matroid M, and prove Theorem 1. The algorithm is given in Algorithm 1 and it
combines the continuous Greedy algorithm of [3] with a discrete Greedy algorithm that we
provide in this paper, building on [5].
Algorithm 1 Algorithm for the continuous problem maxx?P (M) F (x).
1: procedure ContinuousMatroid(f, M, )
2: c0 = ?(1/ ), where the ? hides a sufficiently large absolute constant
3: S = LazySamplingGreedy(f, M, )
4: x = ContinuousGreedy(fS, M/S, c0, )
5: return 1S ? x hh x ? y is the vector (x ? y)i = max{xi, yi} for all i ii
6: end procedure
The continuous Greedy algorithm. The algorithm used on line 4 is the algorithm of [3].
To obtain a fast running time, we use an independent set data structure to maintain the
independent sets constructed by the algorithm. The data structure needs to support two
operations: add an element to the independent set, and check whether an element can be
added to the independent set while maintaining independence. For a partition matroid, such
a data structure with O(1) time per operation is trivial to obtain. For a graphic matroid, we
can use a unionfind data structure [13, 23] with a O(log? k) amortized time per operation.
I Lemma 7 (Corollary 3.1 in [5]; [3]). When run with values c and ? as input,
ContinuousGreedy uses O(n ln(n/?)/?2) independent set data structure operations, and
O(cn ln2(n/?)/?4) queries to the value oracle of f and additional arithmetic operations.
Moreover, if maxS?I Pe?S f (e) ? c ? f (OPT), where OPT ? argmaxS?I f (S), the solution
1
x returned by the algorithm satisfies F (x) ? (1 ? e ? ?)f (OPT).
The discrete Greedy algorithm is given in Algorithm 2. The algorithm works for any
matroid constraint for which we can provide a fast data structure for maintaining a maximum
weight base (note that the base is only an approximate maximum weight base, and we drop
the word approximate for simplicity). We now describe the properties we require from this
data structure. As discussed in the introduction, we give such data structures for a graphic
matroid and a partition matroid in the full version of this paper.
The dynamic maximum weight base data structure. Algorithm 2 makes use of a data
structure for maintaining the maximum weight base in the matroid, where each element has
a weight and the weights are updated through a sequence of updates that can only decrease
the weights. The data structure needs to support the following operation: UpdateBase
decreases the weight of an element and it updates the base to a maximum weight base for
the updated weights. The data structures that we provide in the full version of the paper for
a graphic and a partition matroid support this operation in O(poly(log k)) amortized time.
We note here that the data structure maintains a maximum weight base of the original
matroid M, and not the contracted matroid M/S obtained after picking a set S of elements.
This suffices for us, since the discrete Greedy algorithm that we use will not change the
weight of an element after it was added to the solution S. Due to this invariant, we can show
that the maximum weight base B of M that the data structure maintains has the property
that S ? B at all times, and B \ S is a maximum weight base in M/S. This follows from
the observation that, if an element e is in the maximum weight base B and the only changes
to the weights are such that the weight of e remains unchanged and the weights of elements
other than e are decreased, then e remains in the new maximum weight base.
The discrete Greedy algorithm. The algorithm (Algorithm 2) is based on the random
residual Greedy algorithm of [4]. The latter algorithm constructs a solution S over k iterations.
In each iteration, the algorithm assigns a linear weight to each element that is equal to the
marginal gain f (S ? {e}) ? f (S) on top of the current solution, and it finds a maximum
weight base B in M/S. The algorithm then samples an element of B uniformly at random
and adds it to the solution. As discussed in Section 1.2, the key difficulty for obtaining a fast
running time is maintaining the maximum weight base. Our algorithm uses the following
approach for maintaining an approximate maximum weight base. The algorithm maintains
the marginal value of each element (rounded to the next highest power of (1 ? )), and it
updates it in a lazy manner; at every point, w(e) denotes the cached (rounded) marginal
value of the element, and it may be stale.
The algorithm maintains the base B using the data structure discussed above that
supports the UpdateBase operation. Additionally, the elements of B \ S are stored into
buckets corresponding to geometrically decreasing marginal values. More precisely, there
are N = O(log(k/ )/ ) buckets B(1), B(2), . . . , B(N). The jth bucket B(j) contains all of
the elements of B with marginal values in the range ((1 ? )jM, (1 ? )j?1M ], where M
is a value such that f (OPT) ? M ? O(1)f (OPT) (we assume that the algorithm knows
such a value M , as it can be obtained in nearlylinear time, see e.g. Lemma 3.2 in [5]). The
remaining elements of B that do not appear in any of the N buckets have marginal values
at most (1 ? )N M ; these elements have negligible total marginal gain, and they can be
safely ignored.
In order to achieve a fast running time, after each Greedy step, the algorithm uses
sampling to (partially) update the base B, the cached marginal values, and the buckets. This
is achieved by the procedure RefreshValues, which works as follows. RefreshValues
considers each of the buckets in turn. The algorithm updates the bucket B(j) by spot checking
O(log n) elements sampled uniformly at random from the bucket. For each sampled element
e, the algorithm computes its current marginal value and, if it has decreased below the range
of its bucket, it moves the element to the correct buckets and call UpdateBase to maintain
the invariant that B is a maximum weight base.
When the algorithm finds an element whose bucket has changed, it resets to 0 the count
for the number of samples taken from the bucket. Thus the algorithm keeps sampling from
the bucket until ?(log n) consecutive sampled elements do not change their bucket. The
sampling step ensures that, with high probability, in each bucket at least half of the elements
are in the correct bucket. (We remark that, instead of resetting the sample count to 0, it
suffices to decrease the count by 1, i.e., the count is the total number of samples whose
bucket was correct minus the number of samples whose bucket was incorrect. The algorithm
then stops when this tally reaches ?(log n). This leads to an improvement in the running
time, but we omit it in favor of a simpler analysis.)
After running RefreshValues, the algorithm samples an element e uniformly at random
from B \ S and adds it to S. The algorithm then removes e from the buckets; this ensures
that the weight of e will remain unchanged for the remainder of the algorithm.
2.1
Analysis of the approximation guarantee
Here we show that Algorithm 1 achieves a 1 ? 1/e ? approximation. We first analyze the
LazySamplingGreedy algorithm. We start with some convenient definitions. Consider
some point in the execution of the LazySamplingGreedy algorithm. Consider a bucket
B(j). At this point, each element e ? B(j) is in the correct bucket iff its current marginal
value f (S ? {e}) ? f (S) lies in the interval ((1 ? )jM, (1 ? )j?1M ] (its cached marginal
value w(e) lies in that interval, but it might be stale). We say that the bucket B(j) is good if
at least half of the elements in B(j) are in the correct bucket, and we say that the bucket is
bad otherwise.
The following lemma shows that, with high probability over the random choices of
RefreshValues, each run of RefreshValues ensures that every bucket B(j) with j ? [N ]
is good.
I Lemma 8. Consider an iteration in which LazySamplingGreedy calls RefreshValues.
When RefreshValues terminates, the probability that the buckets {B(j) : j ? [N ]} are all
good is at least 1 ? 1/n2.
Proof. We will show that the probability that a given bucket is bad is at most 5 log n/n3; the
claim then follows by the union bound, since there are N ? n/(5 log n) buckets. Consider a
bucket B(j), where j ? [N ], and suppose that the bucket is bad at the end of RefreshValues.
We analyze the probability the bucket is bad because the algorithm runs until iteration
t, which is the last time the algorithm finds an element in B(j) in the wrong bucket, and
for 4 log n iterations after t, it always find elements in the right bucket even though only
1/2 of B(j) are in the right bucket. Since at most half of the elements of B(j) are in the
correct bucket and the samples are independent, this event happens with probability at most
(1/2)4 log2 n = 1/n4. By the union bound over all choices of t = 1, 2, . . . , 5n log n, the failure
probability for bucket B(j) is at most 5 log n/n3. J
Since LazySamplingGreedy performs at most k ? n iterations, it follows by the union
bound that all of the buckets {B(j) : j ? [N ]} are all good throughout the algorithm with
probability at least 1 ? 1/n. For the remainder of the analysis, we condition on this event.
Additionally, we fix an event specifying the random choices made by RefreshValues and
we implicitly condition all probabilities and expectations on this event.
Let us now show that B is a suitable approximation for the maximum weight base in
M/S with weights given by the current marginal values f (S ? {e}) ? f (S).
I Lemma 9. Suppose that every bucket of B is good throughout the algorithm. Let v(e) =
f (S ? {e}) ? f (S) denote the current marginal values. We have
(1) w(S0) ? v(S0) for every S0 ? V ;
(2) w(B) ? w(S0) for every S0 ? V ;
2
(3) v(B) ? 1?2 ? w(B) ? k ? M .
Proof. The first property follows from the fact that, by submodularity, the weights w(?) are
upper bounds on the marginal values.
The second property follows from the fact that the algorithm maintains the invariant
that B is the maximum weight base in M/S with respect to the weights w(?).
Let us now show the third property. Consider the following partition of B into sets B1,
B2, and B3, where: B1 is the set of all elements e ? B such that e is in one of the buckets
{B(j) : j ? [N ]} and moreover e is in the correct bucket (note that (1 ? )w(e) ? v(e) ? w(e)
for every e ? B1); B2 is the set of all elements e ? B such that e is in one of the buckets
{B(j) : j ? [N ]} but e is not in the correct bucket (i.e., (1 ? )N M < w(e) but v(e) <
(1 ? )w(e)); B3 is the set of all elements e ? B2 such that w(e) < (1 ? )N M ? ( /k)2M .
2
Since B3 ? k, we have w(B3) ? B3 ? k M ? k M.
Since all of the buckets are good, it follows that the total w(?) weight of the elements
that are in the correct bucket is at least 12 ? w(B \ B3). Indeed, we have
N N
w(B1) = X w(B1?B(j)) = X(1? )j?1M B1?B(j) ?
j=1 j=1
j=1
N
X(1? )j?1M B(j) = w(B \ B3) .
2 2
Finally, since v(e) ? (1 ? )w(e) for every e ? B1, we have
Now we turn to the analysis of the main for loop of LazySamplingGreedy (lines
11?19). Let Z be a random variable equal to the number of iterations where the algorithm
executes line 17. We define sets {St : t ? {0, 1, . . . , k}} and {OPTt : t ? {0, 1, . . . , k}} as
follows. Let S0 = ? and OPT0 = OPT. Consider an iteration t ? Z and suppose that
St?1 and OPTt?1 have already been defined and they satisfy St?1 ? OPTt?1 ? I and
St?1 + OPTt?1 = k. Consider a bijection ? : B ? OPTt?1 so that OPTt?1 \ {?(e)} ? {e}
is a base of M/St?1 for all e ? B (such a bijection always exists, see e.g., Corollary 39.12a in
[22]). Let et be the element sampled on line 17 and ot = ?(et). We define St = St?1 ? {et}
and OPTt = OPTt?1 \ {ot}. Note that St ? OPTt ? I. For each t > Z, we define St = SZ
and OPTt = OPTZ .
In each iteration t, the gain in the Greedy solution value is f (St) ? f (St?1), and the loss
in the optimal solution value is f (OPTt?1) ? f (OPTt) (when we add an element to St?1,
we remove an element from OPTt?1 so that St ? OPTt remains a feasible solution). The
following lemma relates the two values in expectation.
I Lemma 10. For every t ? [k], if all of the buckets B(j) are good, we have
E[f (St) ? f (St?1)] ? c ? E[f (OPTt?1) ? f (OPTt)].
Proof. Consider t ? [k]. Recall that Z is the number of iterations where the algorithm
executes line 17. If t > Z, the inequality is trivially satisfied, since both expectations
are equal to 0. Therefore we may assume that t ? Z and thus St = St?1 ? {et} and
OPTt = OPTt?1 \ {ot}.
Let us now fix an event Rt?1 specifying the random choices for the first t ? 1 iterations,
i.e., the random elements e1, . . . , et?1 and o1, . . . , ot?1. In the following, all the probabilities
and expectations are implicitly conditioned on Rt?1. Note that, once Rt?1 is fixed, St?1 and
OPTt?1 are deterministic.
Let us first lower bound E[f (St?1 ? {et}) ? f (St?1)]. Let wt, Bt, and Wt denote w, B,
and W right after executing RefreshValues in iteration t. Note that, since Rt?1 and the
random choices of RefreshValues are fixed, wt, Bt, and Wt are deterministic.
Recall that all of the buckets of Bt are good, i.e., at least half of the elements of B(j) are
t
in the correct bucket, for every j ? [N ]. Let Bt0 be the subset of Bt consisting of all of the
elements that are in the correct bucket, and let Bt00 be the subset of Bt consisting of all of
the elements that are not in any bucket.
For every e ? Bt0, we have f (St?1 ? {e}) ? f (St?1) ? (1 ? )wt(e). For every e ? B00, we
t 2
have f (St?1 ? {e}) ? f (St?1) ? wt(e) = (1 ? )N M ? ( /k)2M, and therefore wt(Bt00) ? k M.
Since all of the buckets are good and Wt > 4cM (the algorithm did not terminate on line 14),
we have
N N
X wt(e) = X wt(Bt0 ? Bt(j)) = X(1 ? )j?1M Bt0 ? Bt(j) ?
e?Bt0 j=1 j=1
2
= wt(Bt \ Bt00)
2
Wt
? 2 ? 2k M ?
By combining these observations, we obtain
E[f (St?1 ? {et}) ? f (St?1)] ? E[f (St?1 ? {et}) ? f (St?1)et ? Bt0] Pr[et ? Bt0]
= E[f (St?1 ? {et}) ? f (St?1)et ? Bt0] ? Bt0 ? (1 ? )E[wt(et)et ? Bt0] ? Bt0
Bt Bt
1 Bt0
= (1 ? )wt(Bt0) ? Bt0 ? Bt ?
2
(1 ? ) 2c ? 2k
Bt
f (OPT) ? Bct f (OPT)
Let us now upper bound E[f (OPTt?1) ? f (OPTt)]. Recall that et is chosen randomly from
B and thus, ot is chosen uniformly at random from OPTt?1 (since ? is a bijection). Hence
E[f (OPTt?1) ? f (OPTt?1 \ {ot})] = X (f (OPTt?1) ? f (OPTt?1 \ {o})) ? OPT1t?1
o?OPTt?1
By submodularity, we have
m
f (OPTt?1) ? X(f (OPTt?1) ? f (OPTt?1 \ {oj}).
j=1
Therefore
E[f (OPTt?1) ? f (OPTt?1 \ {ot})] ?
To recap, we have shown that:
E[f (St?1 ? {et}) ? f (St?1)] ?
Bt
f (OPT)
E[f (OPTt?1) ? f (OPTt?1 \ {ot})] ? OPTt?1 .
c ? f (OPT) ,
Since Bt = OPTt?1, we have
f (OPTt?1)
OPTt?1
f (OPT)
? OPTt?1
.
E[f (St?1 ? {et}) ? f (St?1)] ? c ? E[f (OPTt?1) ? f (OPTt?1 \ {ot})].
Since the above inequality holds conditioned on every given event Rt?1, it holds
unconditionally, and the lemma follows. J
The following lemma follows from Lemmas 9 and 10.
I Lemma 11. If all of the buckets B(j) are good, the LazySamplingGreedy algorithm
(Algorithm 2) returns a set S ? I with the following properties.
(1) maxS0 : S0?S?I Pe?S0 fS(e) ? 4cM = O(1/ )f (OPT).
(2) There is a random subset OPT0 ? OPT depending on S with the following properties:
S ? OPT0 ? I and E[f (OPT0)] ? f (OPT) ? c ? E[f (S)] ? 1 ? 1c f (OPT).
1
By combining Lemmas 7 and 11, we obtain:
I Lemma 12. The ContinuousMatroid algorithm (Algorithm 1) returns a solution
1S ? x ? P (M) such that F (1S ? x) ? (1 ? 1/e ? O( ))f (OPT) with constant probability.
Proof. Note that, in order to apply Lemma 7, we need the following condition to hold:
maxS0 : S0?S?I Pe?S0 fS(e) ? c0fS(OPT00), where OPT00 ? argmaxS0 : S0?S?I fS(S0).
Using Lemma 11, we can show that the above condition holds with constant probability
as follows. Let OPT0 be the set guaranteed by Lemma 11. We have fS(OPT0) ? fS(OPT00)
and f (S ? OPT0) ? f (OPT0). Therefore fS(OPT00) ? fS(OPT0) ? f (OPT0) ? f (S).
By Lemma 11, we have E[f (OPT) ? f (OPT0)] ? f (OPT)/c. Therefore, by the Markov
inequality, with probability at least 2/3, we have f (OPT)?f (OPT0) ? 3f (OPT)/c. Consider
two cases. First, if f (S) ? (1 ? 1/e)f (OPT) then the algorithm can simply return S. Second,
if f (S) < (1 ? 1/e)f (OPT) then fS(OPT00) ? f (OPT0) ? f (S) ? (1/e ? 3/c)f (OPT).
Therefore, maxS0 : S0?S?I Pe?S0 fS(e) ? O(cfS(OPT00)) ? c0fS(OPT00). Thus the conditions
of Lemma 7 are satisfied and thus the continuous Greedy algorithm returns a solution
x ? P (M/S) such that
1
1 ? e ?
F (1S ? x) ? f (S) ?
22
23
24
Alexander Ageev and Maxim Sviridenko . Pipage Rounding: A New Method of Constructing Algorithms with Proven Performance Guarantee . Journal of Combinatorial Optimization , 8 ( 3 ): 307  328 , 2004 .
Yossi Azar and Iftah Gamzu . Efficient Submodular Function Maximization under Linear Packing Constraints . In International Colloquium on Automata, Languages and Programming (ICALP) , 2012 .
Ashwinkumar Badanidiyuru and Jan Vondr?k . Fast algorithms for maximizing submodular functions . In ACMSIAM Symposium on Discrete Algorithms (SODA) , 2014 .
Niv Buchbinder , Moran Feldman, Joseph Naor, and Roy Schwartz . Submodular Maximization with Cardinality Constraints . In ACMSIAM Symposium on Discrete Algorithms (SODA) , 2014 .
Niv Buchbinder , Moran Feldman, and Roy Schwartz . Comparing Apples and Oranges: Query Tradeoff in Submodular Maximization . Math. Oper. Res. , 42 ( 2 ): 308  329 , 2017 .
Gruia Calinescu , Chandra Chekuri, Martin P?l , and Jan Vondr?k . Maximizing a Submodular Set Function Subject to a Matroid Constraint . SIAM Journal on Computing , 40 ( 6 ): 1740  1766 , 2011 .
Chandra Chekuri , T. S. Jayram , and Jan Vondr?k . On Multiplicative Weight Updates for Concave and Submodular Function Maximization . In Conference on Innovations in Theoretical Computer Science (ITCS) , 2015 . doi: 10 .1145/2688073.2688086.
Chandra Chekuri , Jan Vondr?k, and Rico Zenklusen . Dependent Randomized Rounding via Exchange Properties of Combinatorial Structures . In IEEE Foundations of Computer Science (FOCS) , pages 575  584 . IEEE Computer Society, 2010 .
Theory of Computing , 8 ( 1 ): 95  119 , 2012 .
Uriel Feige . A threshold of ln n for approximating set cover . jacm, 45 : 634  652 , 1998 .
Yuval Filmus and Justin Ward . Monotone Submodular Maximization over a Matroid via NonOblivious Local Search . SIAM Journal on Computing , 43 ( 2 ): 514  542 , 2014 .
M L Fisher , G L Nemhauser, and L A Wolsey . An analysis of approximations for maximizing submodular set functionsII . Mathematical Programming Studies , 8 : 73  87 , 1978 .
Bernard A Galler and Michael J Fisher . An improved equivalence algorithm . Communications of the ACM , 7 ( 5 ): 301  303 , 1964 .
Ryan Gomes and Andreas Krause . Budgeted Nonparametric Learning from Data Streams . In International Conference on Machine Learning (ICML) , pages 391  398 , 2010 .
Jacob Holm , Kristian De Lichtenberg, and Mikkel Thorup . Polylogarithmic deterministic fullydynamic algorithms for connectivity, minimum spanning tree, 2edge, and biconnectivity .
Journal of the ACM (JACM) , 48 ( 4 ): 723  760 , 2001 .
David Kempe , Jon M. Kleinberg , and ?va Tardos . Maximizing the spread of influence through a social network . In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) , pages 137  146 , 2003 .
Andreas Krause , Ajit Paul Singh, and Carlos Guestrin . NearOptimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies . Journal of Machine Learning Research , 9 : 235  284 , 2008 .
Hui Lin and Jeff A. Bilmes. Multidocument Summarization via Budgeted Maximization of Submodular Functions . In Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics , pages 912  920 , 2010 .
Baharan Mirzasoleiman , Ashwinkumar Badanidiyuru, Amin Karbasi, Jan Vondr?k, and Andreas Krause . Lazier Than Lazy Greedy . In AAAI Conference on Artificial Intelligence (AAAI) , 2015 .
G L Nemhauser and L A Wolsey . Best Algorithms for Approximating the Maximum of a Submodular Set Function . Mathematics of Operations Research , 3 ( 3 ): 177  188 , 1978 .
G L Nemhauser , L A Wolsey , and M L Fisher . An analysis of approximations for maximizing submodular set functions I. Mathematical Programming , 14 ( 1 ): 265  294 , 1978 .
Alexander Schrijver . Combinatorial Optimization: Polyhedra and Efficiency . Springer, 2003 .
Robert Endre Tarjan . Efficiency of a good but not linear set union algorithm . Journal of the ACM (JACM) , 22 ( 2 ): 215  225 , 1975 .
Jan Vondr?k . Optimal approximation for the submodular welfare problem in the value oracle model . In ACM Symposium on Theory of Computing (STOC) , 2008 .