Towards Nearly-Linear Time Algorithms for Submodular Maximization with a Matroid Constraint

LIPICS - Leibniz International Proceedings in Informatics, Jul 2019

We consider fast algorithms for monotone submodular maximization subject to a matroid constraint. We assume that the matroid is given as input in an explicit form, and the goal is to obtain the best possible running times for important matroids. We develop a new algorithm for a general matroid constraint with a 1 - 1/e - epsilon approximation that achieves a fast running time provided we have a fast data structure for maintaining an approximately maximum weight base in the matroid through a sequence of decrease weight operations. We construct such data structures for graphic matroids and partition matroids, and we obtain the first algorithms for these classes of matroids that achieve a nearly-optimal, 1 - 1/e - epsilon approximation, using a nearly-linear number of function evaluations and arithmetic operations.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

http://drops.dagstuhl.de/opus/volltexte/2019/10630/pdf/LIPIcs-ICALP-2019-54.pdf

Towards Nearly-Linear Time Algorithms for Submodular Maximization with a Matroid Constraint

I C A L P Towards Nearly-Linear Time Algorithms for Submodular Maximization with a Matroid Constraint Huy L. Nguyen 0 1 2 Category Track A: Algorithms, Complexity and Games 0 Alina Ene Department of Computer Science, Boston University , MA , USA 1 Funding Alina Ene: Partially supported by NSF CAREER grant CCF-1750333 and NSF grant CCF-1718342. Huy L. Nguyen: Partially supported by NSF CAREER grant CCF-1750716 2 College of Computer and Information Science, Northeastern University , Boston, MA , USA We consider fast algorithms for monotone submodular maximization subject to a matroid constraint. We assume that the matroid is given as input in an explicit form, and the goal is to obtain the best possible running times for important matroids. We develop a new algorithm for a general matroid constraint with a 1 ? 1/e ? approximation that achieves a fast running time provided we have a fast data structure for maintaining an approximately maximum weight base in the matroid through a sequence of decrease weight operations. We construct such data structures for graphic matroids and partition matroids, and we obtain the first algorithms for these classes of matroids that achieve a nearly-optimal, 1 ? 1/e ? approximation, using a nearly-linear number of function evaluations and arithmetic operations. 2012 ACM Subject Classification Theory of computation ? Submodular optimization and polymatroids and phrases submodular maximization; matroid constraints; fast running times - Acknowledgements This work was done in part while the authors were visiting the Simons Institute for the Theory of Computing. In this paper, we consider fast algorithms for monotone submodular maximization subject to a matroid constraint. Submodular maximization is a central problem in combinatorial optimization that captures several problems of interest, such as maximum coverage, facility location, and welfare maximization. The study of this problem dates back to the seminal work of Nemhauser, Wolsey and Fisher from the 1970?s [20, 21, 12]. Nemhauser et al. introduced a very natural Greedy algorithm for the problem that iteratively builds a solution by selecting the item with the largest marginal gain on top of previously selected items, and they showed that this algorithm achieves a 1 ? 1/e approximation for a cardinality constraint and a 1/2 approximation for a general matroid constraint. The maximum coverage problem is a special case of monotone submodular maximization with a cardinality constraint and it is 1 ? 1/e ET A CS hard to approximate [10], and thus the former result is optimal. Therefore the main question that was left open by the work of Nemhauser et al. is whether one can obtain an optimal, 1 ? 1/e approximation, for a general matroid constraint. In a celebrated line of work [6, 24], Calinescu et al. developed a framework based on continuous optimization and rounding that led to an optimal 1 ? 1/e approximation for the problem. The approach is to turn the discrete optimization problem of maximizing a submodular function f subject to a matroid constraint into a continuous optimization problem where the goal is to maximize the multilinear extension F of f (a continuous function that extends f ) subject to the matroid polytope (a convex polytope whose vertices are the feasible integral solutions). The continuous optimization problem can be solved approximately within a 1 ? 1/e factor using a continuous Greedy algorithm [24] and the resulting fractional solution can be rounded to an integral solution without any loss [1, 6, 8]. The resulting algorithm achieves the optimal 1 ? 1/e approximation in polynomial time. Unfortunately, a significant drawback of this approach is that it leads to very high running times. Obtaining fast running times is a fundamental direction both in theory and in practice, due to the numerous applications of submodular maximization in machine learning, data mining, and economics [18, 17, 14, 16, 9]. This direction has received considerable attention [2, 11, 3, 19, 5, 7], but it remains a significant challenge for almost all matroid constraints. Before discussing these challenges, let us first address the important questions on how the input is represented and how we measure running time. The algorithms in this paper as well as prior work assume that the submodular function is represented as a value oracle that takes as input a set S and returns f (S). For all these algorithms, the number of calls to the value oracle for f dominates the running time of the algorithm (up to a logarithmic factor), and thus we assume for simplicity that each call takes constant time. The algorithms fall into two categories with respect to how the matroid is represented: the independence oracle algorithms assume that the matroid is represented using an oracle that takes as input a set S and returns whether S is feasible (independent); the representable matroid algorithms assume that the matroid is given as input in an explicit form. The representable matroid algorithms can be used for only a subclass of matroids, namely those that can be represented as a linear matroid over vectors in some field1, but this class includes practically-relevant matroids such as the uniform, partition, laminar, graphical, and general linear matroids. The oracle algorithms apply to all matroids, but they are unlikely to lead to the fastest possible running times: even an ideal algorithm that makes only O(k) independence calls has a running time that is ?(k2) in the independence oracle model (each oracle call needs to read its input, which takes ?(k) time in the worst case), even if the matroid is a representable matroid such as a partition or a graphic matroid. This is because Thus there have always been parallel lines of research for representable matroids and general matroids. This work falls in the first category, i.e., we assume that the matroid is given as input in an explicit form, and the goal is to obtain the best possible running times. Note that, although all the representable matroids are linear matroids, it is necessary to consider each class separately, since they have very different running times to even verify if a given solution is feasible: for simple explicit matroids such as a partition or a graphic matroid, checking whether a solution is feasible takes O(n) time, where n is the size of the ground set of the matroid; for general explicit matroids represented using vectors in some field, checking whether a solution is feasible takes O(k?) time, where k is the rank of the matroid and ? is the exponent for fast matrix multiplication. 1 In a linear matroid, the ground set is a collection of n vectors and a subset of the vectors is feasible (independent) if the vectors are linearly independent. Since in many practical settings only nearly-linear running times are feasible, an important question to address is: For which matroid constraints can we obtain a nearly-optimal 1 ? 1/e ? approximation in nearly-linear time? Prior to this work, the only example of such a constraint was a cardinality constraint. For a partition matroid constraint, the fastest running time is ?(n3/2) in the worst case when k = ?(n) [5]. For a graphical matroid constraint, no faster algorithms are known than a general matroid, and the running time is ?(n2) in the worst case when k = ?(n) [3]. Obtaining a best-possible, nearly-linear running time has been very challenging even for these classes of matroids for the following reasons: The continuous optimization is a significant time bottleneck. The continuous optimization problem of maximizing the multilinear extension subject to the matroid polytope is an integral component in all algorithms that achieve a nearly-optimal approximation guarantee. However, the multilinear extension is expensive to evaluate even approximately. To achieve the nearly-optimal approximation guarantees, the evaluation error needs to be very small and in a lot of cases, the error needs to be O(n?1) times the function value. As a result, a single evaluation of the multilinear extension requires ?(n) evaluations of f . Thus, even a very efficient algorithm with O(n) queries to the multilinear extension would require ?(n2) running time. Rounding the fractional solution is a significant time bottleneck as well. Consider a matroid constraint of rank k. The fastest known rounding algorithm is the swap rounding, which requires k swap operations: in each operation, the algorithm has two bases B1 and B2 and needs to find x ? B1, y ? B2 such that B1 \ {x} ? {y} and B2 \ {y} ? {x} are bases. The typical implementation is to pick some x ? B1 and try all y in B2, which requires us to check independence for k solutions. Thus, overall, the rounding algorithm checks independence for ?(k2) solutions. Furthermore, each feasibility check takes ?(k) time just to read the input. Thus a generic rounding for a matroid takes ?(k3) time. Thus, in order to achieve a fast overall running time, one needs fast algorithms for both the continuous optimization and the rounding. In this work, we provide such algorithms for partition and graphic matroids, and we obtain the first algorithms with nearly-linear running times. At the heart of our approach is a general, nearly-linear time reduction that reduces the submodular maximization problem to two data structure problems: maintain an approximately maximum weight base in the matroid through a sequence of decrease-weight operations, and maintain an independent set in the matroid that allows us to check whether an element can be feasibly added. This reduction applies to any representable matroid, and thus it opens the possibility of obtaining faster running times for other classes of matroids. 1.1 Our contributions We now give a more precise description of our contributions. We develop a new algorithm for maximizing the multilinear extension subject to a general matroid constraint with a 1 ? 1/e ? approximation that achieves a fast running time provided we have fast data structures with the following functionality: A maximum weight base data structure: each element has a weight, and the goal is to maintain an approximately maximum weight base in the matroid through a sequence of operations, where each operation can only decrease the weight of a single element; An independent set data structure that maintains an independent set in the matroid and supports two operations: add an element to the independent set, and check whether an element can be added to the independent set while maintaining independence. I Theorem 1. Let f be a monotone submodular function and let M be a matroid on a ground set of size n. Let F be the multilinear extension of f and P (M) be the matroid polytope of M. Suppose that we have a data structure for maintaining a maximum weight base and independent set as described above. There is an algorithm for the problem maxx?P (M) F (x) that achieves a 1 ? 1/e ? approximation using O(n poly(log n, 1/ )) calls to the value oracle for f , data structure operations, and additional arithmetic operations. Using our continuous optimization algorithm and additional results that are included in the full version of this paper, we obtain the first nearly-linear time algorithms for both the discrete and continuous problem with a graphic and a partition matroid constraint. In the graphic matroid case, the maximum weight base data structure is a dynamic maximum weight spanning tree (MST) data structure and the independent data structure is a dynamic connectivity data structure (e.g., union-find), and we can use existing data structures that guarantee a poly-logarithmic amortized time per operation [15, 13, 23]. For a partition matroid, we provide data structures with a constant amortized time per operation. We also address the rounding step and provide a nearly-linear time algorithm for rounding a fractional solution in a graphic matroid. A nearly-linear time rounding algorithm for a partition matroid was provided in [5]. I Theorem 2. There is an algorithm for maximizing a monotone submodular function subject to a generalized partition matroid constraint that achieves a 1 ? 1/e ? approximation using O(n poly(1/ , log n)) function evaluations and arithmetic operations. I Theorem 3. There is an algorithm for maximizing a monotone submodular function subject to a graphic matroid constraint that achieves a 1 ? 1/e ? approximation using O(n poly(1/ , log n)) function evaluations and arithmetic operations. Previously, the best running time for a partition matroid was ?(n3/2 poly(1/ , log n)) in the worst case when k = ?(n) [5]. The previous best running time for a graphic matroid is the same as the general matroid case, which is ?(n2 poly(1/ , log n)) in the worst case when k = ?(n) [3]. As shown by Vondrak [24], there is an efficient reduction from the submodular welfare maximization problem to the submodular maximization problem with a partition matroid constraint. Using this reduction and our algorithm for a partition matroid, we obtain a nearly-linear time algorithm for welfare maximization as well. I Theorem 4. There is a 1 ? 1/e ? approximation algorithm for submodular welfare maximization using O(npoly(1/ , log n)) function evaluations and arithmetic operations. We conclude with a formal statement of the contributions made in this paper on which the results above are based. I Theorem 5. There is a dynamic data structure for maintaining a maximum weight base in a partition matroid through a sequence of decrease weight operations with an O(1) amortized time per operation. I Theorem 6. There is a randomized algorithm based on swap rounding for the graphic matroid polytope that takes as input a point x represented as a convex combination of bases and rounds it to an integral solution S such that E[f (S)] ? F (x). The running time of the algorithm is O(nt log2 n), where t is the number of bases in the convex combination of x. 1.2 Technical overview The starting point of our approach is the work [5]. They observed that the running time of the continuous algorithm using the multilinear extension of [3] depends on the value of the maximum weight base when the value is measured in the modular approximation f 0(S) = Pe?S f (e). It is clear that this approximation is at least the original function and it can be much larger. They observed that the running time is proportional to the ratio between the maximum weight base when weights are measured using the modular approximation compared with the optimal solution when weights are measured using the original function. On the other hand, in the greedy algorithm, the gain in every greedy step is proportional to the maximum weight base when weights are measured using the modular approximation. Thus, the discrete greedy algorithm makes fast progress precisely when the continuous algorithm is slow and vice versa. Therefore, one can start with the discrete greedy algorithm and switch to the continuous algorithm when the maximum weight solution is small even when weights are measured using the modular approximation. Our algorithm consists of two key components: (1) a fast dynamic data structure for maintaining an approximate maximum weight base through a sequence of greedy steps, and (2) an algorithm that makes only a small number of queries to the data structure. Even if fast dynamic data structures are available, previous algorithms including that of [5] cannot achieve a fast time, since they require ?(nk) queries to the data structure: the algorithm of [5] maintains the marginal gain for every element in the current base and it updates them after each greedy step; since each greedy step changes the marginal gain of every element in the base, this approach necessitates ?(k) data structure queries per greedy step. Our new approach uses random sampling to ensure that the number of queries to the data structure is nearly-linear. After each greedy step, our algorithm randomly samples elements from the base to check and update the marginal gains. Because of sampling, it can only ensure that at least 1/2 of the elements in every value range have good estimates of their values. However, this is sufficient for maintaining an approximate maximum weight base. The benefit is that the running time becomes much faster: the number of checks that do not result in updates is small and if we make sure that an update only happens when the marginal gain change by a factor 1 ? then the total number of updates is at most O(n log n/ ). Thus we obtain an algorithm with only a nearly-linear number of data structure queries and additional running time for any matroid constraint. Our approach reduces the algorithmic problem to the data structure problem of maintaining an approximate maximum weight base through a sequence of value updates. In fact, the updates are only decrement in the values and thus can utilize even the decremental data structures as opposed to fully dynamic ones. In the case of a partition matroid constraint, one can develop a simple ad-hoc solution. In the case of a graphic matroid, one can use classical data structures for maintaining minimum spanning trees [15]. In both cases, fast rounding algorithms are also needed. The work [5] gives an algorithm for the partition matroid. We give an algorithm for the graphic matroid based on swap rounding and classical dynamic graph data structures. To obtain fast running time, in each rounding step, instead of swapping a generic pair, we choose a pair involving a leaf of the spanning tree. 1.3 Basic definitions and notation Submodular functions. Let f : 2V ? R+ be a set function on a finite ground set V of size n := |V |. The function is submodular if f (A) + f (B) ? f (A ? B) + f (A ? B) for all subsets A, B ? V . The function is monotone if f (A) ? f (B) for all subsets A ? B ? V . We assume that the function f is given as a value oracle that takes as input any set S ? V and returns f (S). We let F : [0, 1]V ? R+ denote the multilinear extension f . For every x ? [0, 1]V , we have F (x) = X f (S) Y xe Y (1 ? xe) = E[R(x)], S?V e?S e?/S where R(x) is a random set that includes each element e ? V independently with probability xe. Matroids. A matroid M = (V, I) on a ground set V is a collection I of subsets of V , called independent sets, that satisfy certain properties (see e.g., [22], Chapter 39). In this paper, we consider matroids that are given to the input to the algorithm. Of particular interest are the partition and graphic matroids. A generalized partition matroid is defined as follows. We are given a partition V1, V2, . . . , Vh of V into disjoint subsets and budgets k1, k2, . . . , kh. A set S is independent (S ? I) if |S ? Vi| ? ki for all i ? [h]. We let k = Ph i=1 ki denote the rank of the matroid. A graphic matroid is defined as follows. We are given a connected graph on k + 1 vertices and n edges. The independent sets of the matroid are the forests of this graph. Additional notation. Given a set S ? I, we let fS denote the function fS : 2V \S ? R?0 such that fS (S0) = f (S0 ? S) ? f (S) for all S0 ? V \ S. We let M/S = (V \ S, I0) denote the matroid obtained by contracting S in M, i.e., S0 ? I0 iff S0 ? S ? I. We let P (M) denote the matroid polytope of M: P (M) is the convex hull of the indicator vectors of the bases of M, where a base is an independent set of maximum size. Constant factor approximation to f (OPT). Our algorithm needs a O(1) approximation to f (OPT). Such an approximation can be computed very efficiently (see e.g. [5], Lemma 3.2). 1.4 Paper organization In Section 2, we describe our algorithm for the continuous optimization problem of maximizing the multilinear extension subject to a general matroid constraint, with the properties stated in Theorem 1. As discussed in the introduction, our algorithm uses certain data structures to achieve a fast running time. In the full version of the paper, we show how to obtain these data structures for partition and graphic matroids. By combining these data structures with the results of Section 2, we obtain nearly-linear time algorithms for the continuous problem of maximizing the multilinear extension subject to a partition and graphic matroid constraint. To obtain a fast algorithm for the discrete problem, we also need a fast algorithm to round the fractional solution. Buchbinder et al. [5] give a nearly-linear time rounding algorithm for a partition matroid. In the full version of the paper, we give a nearly-linear time rounding algorithm for a graphic matroid, and prove Theorem 6. These results together give Theorems 2 and 3. 2 The algorithm for the continuous optimization problem In this section, we describe and analyze our algorithm for the problem maxx?P (M) F (x) for a general matroid M, and prove Theorem 1. The algorithm is given in Algorithm 1 and it combines the continuous Greedy algorithm of [3] with a discrete Greedy algorithm that we provide in this paper, building on [5]. Algorithm 1 Algorithm for the continuous problem maxx?P (M) F (x). 1: procedure ContinuousMatroid(f, M, ) 2: c0 = ?(1/ ), where the ? hides a sufficiently large absolute constant 3: S = LazySamplingGreedy(f, M, ) 4: x = ContinuousGreedy(fS, M/S, c0, ) 5: return 1S ? x hh x ? y is the vector (x ? y)i = max{xi, yi} for all i ii 6: end procedure The continuous Greedy algorithm. The algorithm used on line 4 is the algorithm of [3]. To obtain a fast running time, we use an independent set data structure to maintain the independent sets constructed by the algorithm. The data structure needs to support two operations: add an element to the independent set, and check whether an element can be added to the independent set while maintaining independence. For a partition matroid, such a data structure with O(1) time per operation is trivial to obtain. For a graphic matroid, we can use a union-find data structure [13, 23] with a O(log? k) amortized time per operation. I Lemma 7 (Corollary 3.1 in [5]; [3]). When run with values c and ? as input, ContinuousGreedy uses O(n ln(n/?)/?2) independent set data structure operations, and O(cn ln2(n/?)/?4) queries to the value oracle of f and additional arithmetic operations. Moreover, if maxS?I Pe?S f (e) ? c ? f (OPT), where OPT ? argmaxS?I f (S), the solution 1 x returned by the algorithm satisfies F (x) ? (1 ? e ? ?)f (OPT). The discrete Greedy algorithm is given in Algorithm 2. The algorithm works for any matroid constraint for which we can provide a fast data structure for maintaining a maximum weight base (note that the base is only an approximate maximum weight base, and we drop the word approximate for simplicity). We now describe the properties we require from this data structure. As discussed in the introduction, we give such data structures for a graphic matroid and a partition matroid in the full version of this paper. The dynamic maximum weight base data structure. Algorithm 2 makes use of a data structure for maintaining the maximum weight base in the matroid, where each element has a weight and the weights are updated through a sequence of updates that can only decrease the weights. The data structure needs to support the following operation: UpdateBase decreases the weight of an element and it updates the base to a maximum weight base for the updated weights. The data structures that we provide in the full version of the paper for a graphic and a partition matroid support this operation in O(poly(log k)) amortized time. We note here that the data structure maintains a maximum weight base of the original matroid M, and not the contracted matroid M/S obtained after picking a set S of elements. This suffices for us, since the discrete Greedy algorithm that we use will not change the weight of an element after it was added to the solution S. Due to this invariant, we can show that the maximum weight base B of M that the data structure maintains has the property that S ? B at all times, and B \ S is a maximum weight base in M/S. This follows from the observation that, if an element e is in the maximum weight base B and the only changes to the weights are such that the weight of e remains unchanged and the weights of elements other than e are decreased, then e remains in the new maximum weight base. The discrete Greedy algorithm. The algorithm (Algorithm 2) is based on the random residual Greedy algorithm of [4]. The latter algorithm constructs a solution S over k iterations. In each iteration, the algorithm assigns a linear weight to each element that is equal to the marginal gain f (S ? {e}) ? f (S) on top of the current solution, and it finds a maximum weight base B in M/S. The algorithm then samples an element of B uniformly at random and adds it to the solution. As discussed in Section 1.2, the key difficulty for obtaining a fast running time is maintaining the maximum weight base. Our algorithm uses the following approach for maintaining an approximate maximum weight base. The algorithm maintains the marginal value of each element (rounded to the next highest power of (1 ? )), and it updates it in a lazy manner; at every point, w(e) denotes the cached (rounded) marginal value of the element, and it may be stale. The algorithm maintains the base B using the data structure discussed above that supports the UpdateBase operation. Additionally, the elements of B \ S are stored into buckets corresponding to geometrically decreasing marginal values. More precisely, there are N = O(log(k/ )/ ) buckets B(1), B(2), . . . , B(N). The j-th bucket B(j) contains all of the elements of B with marginal values in the range ((1 ? )jM, (1 ? )j?1M ], where M is a value such that f (OPT) ? M ? O(1)f (OPT) (we assume that the algorithm knows such a value M , as it can be obtained in nearly-linear time, see e.g. Lemma 3.2 in [5]). The remaining elements of B that do not appear in any of the N buckets have marginal values at most (1 ? )N M ; these elements have negligible total marginal gain, and they can be safely ignored. In order to achieve a fast running time, after each Greedy step, the algorithm uses sampling to (partially) update the base B, the cached marginal values, and the buckets. This is achieved by the procedure RefreshValues, which works as follows. RefreshValues considers each of the buckets in turn. The algorithm updates the bucket B(j) by spot checking O(log n) elements sampled uniformly at random from the bucket. For each sampled element e, the algorithm computes its current marginal value and, if it has decreased below the range of its bucket, it moves the element to the correct buckets and call UpdateBase to maintain the invariant that B is a maximum weight base. When the algorithm finds an element whose bucket has changed, it resets to 0 the count for the number of samples taken from the bucket. Thus the algorithm keeps sampling from the bucket until ?(log n) consecutive sampled elements do not change their bucket. The sampling step ensures that, with high probability, in each bucket at least half of the elements are in the correct bucket. (We remark that, instead of resetting the sample count to 0, it suffices to decrease the count by 1, i.e., the count is the total number of samples whose bucket was correct minus the number of samples whose bucket was incorrect. The algorithm then stops when this tally reaches ?(log n). This leads to an improvement in the running time, but we omit it in favor of a simpler analysis.) After running RefreshValues, the algorithm samples an element e uniformly at random from B \ S and adds it to S. The algorithm then removes e from the buckets; this ensures that the weight of e will remain unchanged for the remainder of the algorithm. 2.1 Analysis of the approximation guarantee Here we show that Algorithm 1 achieves a 1 ? 1/e ? approximation. We first analyze the LazySamplingGreedy algorithm. We start with some convenient definitions. Consider some point in the execution of the LazySamplingGreedy algorithm. Consider a bucket B(j). At this point, each element e ? B(j) is in the correct bucket iff its current marginal value f (S ? {e}) ? f (S) lies in the interval ((1 ? )jM, (1 ? )j?1M ] (its cached marginal value w(e) lies in that interval, but it might be stale). We say that the bucket B(j) is good if at least half of the elements in B(j) are in the correct bucket, and we say that the bucket is bad otherwise. The following lemma shows that, with high probability over the random choices of RefreshValues, each run of RefreshValues ensures that every bucket B(j) with j ? [N ] is good. I Lemma 8. Consider an iteration in which LazySamplingGreedy calls RefreshValues. When RefreshValues terminates, the probability that the buckets {B(j) : j ? [N ]} are all good is at least 1 ? 1/n2. Proof. We will show that the probability that a given bucket is bad is at most 5 log n/n3; the claim then follows by the union bound, since there are N ? n/(5 log n) buckets. Consider a bucket B(j), where j ? [N ], and suppose that the bucket is bad at the end of RefreshValues. We analyze the probability the bucket is bad because the algorithm runs until iteration t, which is the last time the algorithm finds an element in B(j) in the wrong bucket, and for 4 log n iterations after t, it always find elements in the right bucket even though only 1/2 of B(j) are in the right bucket. Since at most half of the elements of B(j) are in the correct bucket and the samples are independent, this event happens with probability at most (1/2)4 log2 n = 1/n4. By the union bound over all choices of t = 1, 2, . . . , 5n log n, the failure probability for bucket B(j) is at most 5 log n/n3. J Since LazySamplingGreedy performs at most k ? n iterations, it follows by the union bound that all of the buckets {B(j) : j ? [N ]} are all good throughout the algorithm with probability at least 1 ? 1/n. For the remainder of the analysis, we condition on this event. Additionally, we fix an event specifying the random choices made by RefreshValues and we implicitly condition all probabilities and expectations on this event. Let us now show that B is a suitable approximation for the maximum weight base in M/S with weights given by the current marginal values f (S ? {e}) ? f (S). I Lemma 9. Suppose that every bucket of B is good throughout the algorithm. Let v(e) = f (S ? {e}) ? f (S) denote the current marginal values. We have (1) w(S0) ? v(S0) for every S0 ? V ; (2) w(B) ? w(S0) for every S0 ? V ; 2 (3) v(B) ? 1?2 ? w(B) ? k ? M . Proof. The first property follows from the fact that, by submodularity, the weights w(?) are upper bounds on the marginal values. The second property follows from the fact that the algorithm maintains the invariant that B is the maximum weight base in M/S with respect to the weights w(?). Let us now show the third property. Consider the following partition of B into sets B1, B2, and B3, where: B1 is the set of all elements e ? B such that e is in one of the buckets {B(j) : j ? [N ]} and moreover e is in the correct bucket (note that (1 ? )w(e) ? v(e) ? w(e) for every e ? B1); B2 is the set of all elements e ? B such that e is in one of the buckets {B(j) : j ? [N ]} but e is not in the correct bucket (i.e., (1 ? )N M < w(e) but v(e) < (1 ? )w(e)); B3 is the set of all elements e ? B2 such that w(e) < (1 ? )N M ? ( /k)2M . 2 Since |B3| ? k, we have w(B3) ? |B3| ? k M ? k M. Since all of the buckets are good, it follows that the total w(?) weight of the elements that are in the correct bucket is at least 12 ? w(B \ B3). Indeed, we have N N w(B1) = X w(B1?B(j)) = X(1? )j?1M |B1?B(j)| ? j=1 j=1 j=1 N X(1? )j?1M |B(j)| = w(B \ B3) . 2 2 Finally, since v(e) ? (1 ? )w(e) for every e ? B1, we have Now we turn to the analysis of the main for loop of LazySamplingGreedy (lines 11?19). Let Z be a random variable equal to the number of iterations where the algorithm executes line 17. We define sets {St : t ? {0, 1, . . . , k}} and {OPTt : t ? {0, 1, . . . , k}} as follows. Let S0 = ? and OPT0 = OPT. Consider an iteration t ? Z and suppose that St?1 and OPTt?1 have already been defined and they satisfy St?1 ? OPTt?1 ? I and |St?1| + |OPTt?1| = k. Consider a bijection ? : B ? OPTt?1 so that OPTt?1 \ {?(e)} ? {e} is a base of M/St?1 for all e ? B (such a bijection always exists, see e.g., Corollary 39.12a in [22]). Let et be the element sampled on line 17 and ot = ?(et). We define St = St?1 ? {et} and OPTt = OPTt?1 \ {ot}. Note that St ? OPTt ? I. For each t > Z, we define St = SZ and OPTt = OPTZ . In each iteration t, the gain in the Greedy solution value is f (St) ? f (St?1), and the loss in the optimal solution value is f (OPTt?1) ? f (OPTt) (when we add an element to St?1, we remove an element from OPTt?1 so that St ? OPTt remains a feasible solution). The following lemma relates the two values in expectation. I Lemma 10. For every t ? [k], if all of the buckets B(j) are good, we have E[f (St) ? f (St?1)] ? c ? E[f (OPTt?1) ? f (OPTt)]. Proof. Consider t ? [k]. Recall that Z is the number of iterations where the algorithm executes line 17. If t > Z, the inequality is trivially satisfied, since both expectations are equal to 0. Therefore we may assume that t ? Z and thus St = St?1 ? {et} and OPTt = OPTt?1 \ {ot}. Let us now fix an event Rt?1 specifying the random choices for the first t ? 1 iterations, i.e., the random elements e1, . . . , et?1 and o1, . . . , ot?1. In the following, all the probabilities and expectations are implicitly conditioned on Rt?1. Note that, once Rt?1 is fixed, St?1 and OPTt?1 are deterministic. Let us first lower bound E[f (St?1 ? {et}) ? f (St?1)]. Let wt, Bt, and Wt denote w, B, and W right after executing RefreshValues in iteration t. Note that, since Rt?1 and the random choices of RefreshValues are fixed, wt, Bt, and Wt are deterministic. Recall that all of the buckets of Bt are good, i.e., at least half of the elements of B(j) are t in the correct bucket, for every j ? [N ]. Let Bt0 be the subset of Bt consisting of all of the elements that are in the correct bucket, and let Bt00 be the subset of Bt consisting of all of the elements that are not in any bucket. For every e ? Bt0, we have f (St?1 ? {e}) ? f (St?1) ? (1 ? )wt(e). For every e ? B00, we t 2 have f (St?1 ? {e}) ? f (St?1) ? wt(e) = (1 ? )N M ? ( /k)2M, and therefore wt(Bt00) ? k M. Since all of the buckets are good and Wt > 4cM (the algorithm did not terminate on line 14), we have N N X wt(e) = X wt(Bt0 ? Bt(j)) = X(1 ? )j?1M |Bt0 ? Bt(j)| ? e?Bt0 j=1 j=1 2 = wt(Bt \ Bt00) 2 Wt ? 2 ? 2k M ? By combining these observations, we obtain E[f (St?1 ? {et}) ? f (St?1)] ? E[f (St?1 ? {et}) ? f (St?1)|et ? Bt0] Pr[et ? Bt0] = E[f (St?1 ? {et}) ? f (St?1)|et ? Bt0] ? |Bt0| ? (1 ? )E[wt(et)|et ? Bt0] ? |Bt0| |Bt| |Bt| 1 |Bt0| = (1 ? )wt(Bt0) ? |Bt0| ? |Bt| ? 2 (1 ? ) 2c ? 2k |Bt| f (OPT) ? |Bct| f (OPT) Let us now upper bound E[f (OPTt?1) ? f (OPTt)]. Recall that et is chosen randomly from B and thus, ot is chosen uniformly at random from OPTt?1 (since ? is a bijection). Hence E[f (OPTt?1) ? f (OPTt?1 \ {ot})] = X (f (OPTt?1) ? f (OPTt?1 \ {o})) ? |OPT1t?1| o?OPTt?1 By submodularity, we have m f (OPTt?1) ? X(f (OPTt?1) ? f (OPTt?1 \ {oj}). j=1 Therefore E[f (OPTt?1) ? f (OPTt?1 \ {ot})] ? To recap, we have shown that: E[f (St?1 ? {et}) ? f (St?1)] ? |Bt| f (OPT) E[f (OPTt?1) ? f (OPTt?1 \ {ot})] ? |OPTt?1| . c ? f (OPT) , Since |Bt| = |OPTt?1|, we have f (OPTt?1) |OPTt?1| f (OPT) ? |OPTt?1| . E[f (St?1 ? {et}) ? f (St?1)] ? c ? E[f (OPTt?1) ? f (OPTt?1 \ {ot})]. Since the above inequality holds conditioned on every given event Rt?1, it holds unconditionally, and the lemma follows. J The following lemma follows from Lemmas 9 and 10. I Lemma 11. If all of the buckets B(j) are good, the LazySamplingGreedy algorithm (Algorithm 2) returns a set S ? I with the following properties. (1) maxS0 : S0?S?I Pe?S0 fS(e) ? 4cM = O(1/ )f (OPT). (2) There is a random subset OPT0 ? OPT depending on S with the following properties: S ? OPT0 ? I and E[f (OPT0)] ? f (OPT) ? c ? E[f (S)] ? 1 ? 1c f (OPT). 1 By combining Lemmas 7 and 11, we obtain: I Lemma 12. The ContinuousMatroid algorithm (Algorithm 1) returns a solution 1S ? x ? P (M) such that F (1S ? x) ? (1 ? 1/e ? O( ))f (OPT) with constant probability. Proof. Note that, in order to apply Lemma 7, we need the following condition to hold: maxS0 : S0?S?I Pe?S0 fS(e) ? c0fS(OPT00), where OPT00 ? argmaxS0 : S0?S?I fS(S0). Using Lemma 11, we can show that the above condition holds with constant probability as follows. Let OPT0 be the set guaranteed by Lemma 11. We have fS(OPT0) ? fS(OPT00) and f (S ? OPT0) ? f (OPT0). Therefore fS(OPT00) ? fS(OPT0) ? f (OPT0) ? f (S). By Lemma 11, we have E[f (OPT) ? f (OPT0)] ? f (OPT)/c. Therefore, by the Markov inequality, with probability at least 2/3, we have f (OPT)?f (OPT0) ? 3f (OPT)/c. Consider two cases. First, if f (S) ? (1 ? 1/e)f (OPT) then the algorithm can simply return S. Second, if f (S) < (1 ? 1/e)f (OPT) then fS(OPT00) ? f (OPT0) ? f (S) ? (1/e ? 3/c)f (OPT). Therefore, maxS0 : S0?S?I Pe?S0 fS(e) ? O(cfS(OPT00)) ? c0fS(OPT00). Thus the conditions of Lemma 7 are satisfied and thus the continuous Greedy algorithm returns a solution x ? P (M/S) such that 1 1 ? e ? F (1S ? x) ? f (S) ? 22 23 24 Alexander Ageev and Maxim Sviridenko . Pipage Rounding: A New Method of Constructing Algorithms with Proven Performance Guarantee . Journal of Combinatorial Optimization , 8 ( 3 ): 307 - 328 , 2004 . Yossi Azar and Iftah Gamzu . Efficient Submodular Function Maximization under Linear Packing Constraints . In International Colloquium on Automata, Languages and Programming (ICALP) , 2012 . Ashwinkumar Badanidiyuru and Jan Vondr?k . Fast algorithms for maximizing submodular functions . In ACM-SIAM Symposium on Discrete Algorithms (SODA) , 2014 . Niv Buchbinder , Moran Feldman, Joseph Naor, and Roy Schwartz . Submodular Maximization with Cardinality Constraints . In ACM-SIAM Symposium on Discrete Algorithms (SODA) , 2014 . Niv Buchbinder , Moran Feldman, and Roy Schwartz . Comparing Apples and Oranges: Query Trade-off in Submodular Maximization . Math. Oper. Res. , 42 ( 2 ): 308 - 329 , 2017 . Gruia Calinescu , Chandra Chekuri, Martin P?l , and Jan Vondr?k . Maximizing a Submodular Set Function Subject to a Matroid Constraint . SIAM Journal on Computing , 40 ( 6 ): 1740 - 1766 , 2011 . Chandra Chekuri , T. S. Jayram , and Jan Vondr?k . On Multiplicative Weight Updates for Concave and Submodular Function Maximization . In Conference on Innovations in Theoretical Computer Science (ITCS) , 2015 . doi: 10 .1145/2688073.2688086. Chandra Chekuri , Jan Vondr?k, and Rico Zenklusen . Dependent Randomized Rounding via Exchange Properties of Combinatorial Structures . In IEEE Foundations of Computer Science (FOCS) , pages 575 - 584 . IEEE Computer Society, 2010 . Theory of Computing , 8 ( 1 ): 95 - 119 , 2012 . Uriel Feige . A threshold of ln n for approximating set cover . jacm, 45 : 634 - 652 , 1998 . Yuval Filmus and Justin Ward . Monotone Submodular Maximization over a Matroid via Non-Oblivious Local Search . SIAM Journal on Computing , 43 ( 2 ): 514 - 542 , 2014 . M L Fisher , G L Nemhauser, and L A Wolsey . An analysis of approximations for maximizing submodular set functions-II . Mathematical Programming Studies , 8 : 73 - 87 , 1978 . Bernard A Galler and Michael J Fisher . An improved equivalence algorithm . Communications of the ACM , 7 ( 5 ): 301 - 303 , 1964 . Ryan Gomes and Andreas Krause . Budgeted Nonparametric Learning from Data Streams . In International Conference on Machine Learning (ICML) , pages 391 - 398 , 2010 . Jacob Holm , Kristian De Lichtenberg, and Mikkel Thorup . Poly-logarithmic deterministic fully-dynamic algorithms for connectivity, minimum spanning tree, 2-edge, and biconnectivity . Journal of the ACM (JACM) , 48 ( 4 ): 723 - 760 , 2001 . David Kempe , Jon M. Kleinberg , and ?va Tardos . Maximizing the spread of influence through a social network . In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) , pages 137 - 146 , 2003 . Andreas Krause , Ajit Paul Singh, and Carlos Guestrin . Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies . Journal of Machine Learning Research , 9 : 235 - 284 , 2008 . Hui Lin and Jeff A. Bilmes. Multi-document Summarization via Budgeted Maximization of Submodular Functions . In Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics , pages 912 - 920 , 2010 . Baharan Mirzasoleiman , Ashwinkumar Badanidiyuru, Amin Karbasi, Jan Vondr?k, and Andreas Krause . Lazier Than Lazy Greedy . In AAAI Conference on Artificial Intelligence (AAAI) , 2015 . G L Nemhauser and L A Wolsey . Best Algorithms for Approximating the Maximum of a Submodular Set Function . Mathematics of Operations Research , 3 ( 3 ): 177 - 188 , 1978 . G L Nemhauser , L A Wolsey , and M L Fisher . An analysis of approximations for maximizing submodular set functions- I. Mathematical Programming , 14 ( 1 ): 265 - 294 , 1978 . Alexander Schrijver . Combinatorial Optimization: Polyhedra and Efficiency . Springer, 2003 . Robert Endre Tarjan . Efficiency of a good but not linear set union algorithm . Journal of the ACM (JACM) , 22 ( 2 ): 215 - 225 , 1975 . Jan Vondr?k . Optimal approximation for the submodular welfare problem in the value oracle model . In ACM Symposium on Theory of Computing (STOC) , 2008 .


This is a preview of a remote PDF: http://drops.dagstuhl.de/opus/volltexte/2019/10630/pdf/LIPIcs-ICALP-2019-54.pdf

Alina Ene, Huy L. Nguyen. Towards Nearly-Linear Time Algorithms for Submodular Maximization with a Matroid Constraint, LIPICS - Leibniz International Proceedings in Informatics, 2019, 54:1-54:14, DOI: 10.4230/LIPIcs.ICALP.2019.54