|
| 1 | +--- |
| 2 | +tags: |
| 3 | + - OMSCS |
| 4 | + - Algorithms |
| 5 | +--- |
| 6 | +# 06.2 - NP - Definitions |
| 7 | +> How do we prove that a problem is computationally difficult? |
| 8 | +
|
| 9 | + |
| 10 | +## Outline |
| 11 | +- NP |
| 12 | +- NP-Complete |
| 13 | +- Reductions |
| 14 | +- NP-Completeness |
| 15 | + - 3SAT |
| 16 | + - Graph Problems |
| 17 | + - Knapsack |
| 18 | +- What's NP-completeness mean? |
| 19 | +- What's $P=NP$ and $P\ne NP$ |
| 20 | +- How do we show that a problem is intractable? |
| 21 | + - Intractable - unlikely to be solvable efficiently |
| 22 | + |
| 23 | +## Complexity Classes |
| 24 | +- NP = the class of all **search** problems |
| 25 | + - can use **decision** problems |
| 26 | + - Forget about the notion of decision. Focus on search. |
| 27 | +- P = the class of search problems that are solvable in polynomial time |
| 28 | +- P is a subset of NP ($P \subset NP$) |
| 29 | +- Rough definition of search problem |
| 30 | + - A problem where we can efficiently verify solutions |
| 31 | + - "efficiently" = verifiable in polynomial time |
| 32 | +- What's all this $P=NP$ vs $P \ne NP$ business? |
| 33 | + - $P$ - solving a problem in polynomial time |
| 34 | + - $NP$ - verifying a solution in polynomial time |
| 35 | + - Are these the same set of problems? |
| 36 | + - Is the process of verifying a solution in polynomial time the same as the process of generating that solution in polynomial time? |
| 37 | + |
| 38 | +## Search Problems |
| 39 | +**Form:** Given instance $I$, find a solution $S$ for $I$ if one exists. Output NO if $I$ has no solutions. |
| 40 | + |
| 41 | +**Requirement:** To be a search problem, if given an instance $I$ and solution $S$, then we can verity that $S$ is a solution to $I$ in polynomial time (wrt $|I|$). |
| 42 | + |
| 43 | +We only have to verify solutions when we're given a solution. |
| 44 | + |
| 45 | +## Satisfiability Problem (SAT) |
| 46 | +- This is one of the original NP-complete problems |
| 47 | +- **Input:** Boolean formula $f$ in CNF, with $n$ variables and $m$ clauses. |
| 48 | +- **Output:** satisfying assignment if one exists, and NO (unsat.) otherwise. |
| 49 | + |
| 50 | +Example: |
| 51 | +- $f=(x_3 \vee \overline{x_2} \vee \overline{x_1}) \wedge (x_1) \wedge (x_2 \vee \overline{x_3}) \wedge (\overline{x_1} \vee \overline{x_3})$ |
| 52 | +- Clause 2 forces $x_1=T$ |
| 53 | +- Clause 4 forces $x_3=F$ |
| 54 | +- Clause 3 is redundant. |
| 55 | +- Clause 1 forces $x_2=F$ |
| 56 | + |
| 57 | +Given $f$ and assignment of T/F to $x_1,x_2,...,x_n$, what is the running time to verify? |
| 58 | +- There are $n$ variables. |
| 59 | +- There are $m$ clauses. |
| 60 | +- Each clause is made of some number of literals. This can be unbounded, based on the definitions provided in this lecture so far. |
| 61 | +- For each of $m$ clauses, we need to iterate over the unique set of literals that comprise the clause and see if any of the literals are True. There are at most 2n unique literals in each clause. |
| 62 | +- $O(nm)$ |
| 63 | +- Therefore SAT $\in NP$ |
| 64 | + |
| 65 | +## Colorings in NP |
| 66 | +- The k-colorings problem |
| 67 | +- **Input:** |
| 68 | + - undirected $G=(V,E)$ |
| 69 | + - integer $k \gt 0$. $k$ is the number of colors. |
| 70 | +- **Output:** |
| 71 | + - Assign each vertex a color in $\{1, 2, ..., k\}$ |
| 72 | + - Adjacent vertices must have different colors. |
| 73 | + - Return NO if no such $k$-coloring exists for $G$. |
| 74 | +- k-colorings is in NP. |
| 75 | + - For each vertex ($v \in V$) |
| 76 | + - check all neighboring vertices ($u \in V$) |
| 77 | + - If any have the same color as $v$, the k-coloring is incorrect. |
| 78 | + - Classic $O(n+m):n=|V|,m=|E|$ linear graph algorithm |
| 79 | + - The lectures state that this is $O(m)$ time, which makes sense. You don't need to check vertices which have no neighbors. This depends on your graph data structuring. |
| 80 | + |
| 81 | +## MST |
| 82 | +- **Input:** $G=(V,E)$ with positive edge lengths. |
| 83 | +- **Output:** tree $T$ with minimum weight. |
| 84 | +- This problem is in both $P$ and $NP$ |
| 85 | + - MST $\in NP$ ? |
| 86 | + - Can we validate an MST in poly-time? |
| 87 | + - We can run BFS/DFS to check that $T$ is a tree that spans $G$ |
| 88 | + - We can generate a new MST with a known correct algorithm (K's or P's) and compare the total weight against $T$. |
| 89 | + - $O(m \space log \space n) < O(mn) \le O(n^3)$ |
| 90 | + - MST $\in NP$ ✅ |
| 91 | + - MST $\in P$ ? |
| 92 | + - We first need to demonstrate that MST is a search problem. This was performed by validating that MST $\in NP$. |
| 93 | + - Running Kruskal's/Prim's: $O(m \space log \space n) < O(mn) \le O(n^3)$ |
| 94 | + - This is polynomial time for generating a solution. |
| 95 | + - MST $\in P$ ✅ |
| 96 | + |
| 97 | +## Knapsack Problem |
| 98 | +- [[02.1 - Dynamic Programming 2 - Knapsack]] |
| 99 | +- **Input:** |
| 100 | + - $n$ objects |
| 101 | + - integer weights: $w_1,...,w_n$ |
| 102 | + - integer values: $v_1,...,v_n$ |
| 103 | + - capacity $B$ |
| 104 | +- **Output:** |
| 105 | + - subset $S$ of objects with total weight $\le B$ ($\sum_{i \in S} w_i \le B$) |
| 106 | + - maximum total value ($\max \{\sum_{i \in S}v_i\}$) |
| 107 | + - Variants allow or disallow repetition. |
| 108 | +- Knapsack $\in NP$ ? |
| 109 | + - Given an instance $I=\{w_1, ..., w_n, v_1,...,v_n,B\}$ |
| 110 | + - Given a solution $S$ |
| 111 | + - We need to check solution $S$ |
| 112 | + - Validate that $\sum_{i \in S} w_i \le B$. This is $O(n)$ time. |
| 113 | + - How do we validate that $S$ maximizes $\sum_{i \in S}v_i$? |
| 114 | + - This is the whole problem. We can't run the DP algorithm to validate, because the DP algorithm is exponential. |
| 115 | + - We currently are unable to prove (as a society) whether knapsack is **definitively in** NP or **definitively NOT in** NP. |
| 116 | + - "Knapsack is not known to be in NP." |
| 117 | +- Knapsack $\in P$ ? |
| 118 | + - No, the best algorithms for generating an exact solution are $O(nB)$ |
| 119 | + - $B$ is represented in $log(B)$ bits. |
| 120 | + - Therefore $O(nB)$ is **exponential** in the input size. |
| 121 | + |
| 122 | +## Knapsack Search |
| 123 | +- This is a variant of Knapsack which is in NP |
| 124 | +- **Input:** |
| 125 | + - $n$ objects |
| 126 | + - integer weights: $w_1,...,w_n$ |
| 127 | + - integer values: $v_1,...,v_n$ |
| 128 | + - capacity $B$ |
| 129 | + - **Plus a goal $g$** |
| 130 | +- **Output:** |
| 131 | + - subset $S$ of objects with total weight $\le B$ ($\sum_{i \in S} w_i \le B$) |
| 132 | + - total value meets goal ($\sum_{i \in S}v_i \ge g$) |
| 133 | + - Variants allow or disallow repetition. |
| 134 | + - Returns NO if no such $S$ exists which meets the goal. |
| 135 | +- Knapsack Search $\in NP$ ? |
| 136 | + - Given an instance $I=\{w_1, ..., w_n, v_1,...,v_n,B,g\}$ |
| 137 | + - Given a solution $S$ |
| 138 | + - We need to check solution $S$ |
| 139 | + - We don't need to validate the "NO" result. |
| 140 | + - Validate that $\sum_{i \in S} w_i \le B$. This is $O(n)$ time. |
| 141 | + - Validate that $\sum_{i \in S}v_i \ge g$. This is also $O(n)$ time. |
| 142 | + - Knapsack Search $\in NP$ ? ✅ |
| 143 | +- Some additional notes on verification |
| 144 | + - The size of W is $\sum_{i=1}^n w_i$ |
| 145 | + - The size of V is $\sum_{i=1}^n v_i$ |
| 146 | + - The time required to compute $\sum_{i \in S} w_i \le B$ is at most $O(n \space log \space W)$ |
| 147 | + - The time required to compute $\sum_{i \in S}v_i \ge g$ is at most $O(n \space log \space V)$ |
| 148 | + - The $log$ terms are effectively showing the amount of time required to add all of the bits in all of the numbers. |
| 149 | + - These are still both polynomial in the input size, even for arbitrarily large integers. |
| 150 | + |
| 151 | +## Terminology |
| 152 | +- $P$ stands for "polynomial time" |
| 153 | +- $NP$ stands for "nondeterministic polynomial time" |
| 154 | + - This the class of problems that can be solved in polynomial time on a **non-deterministic** machine. |
| 155 | + - These are machines that are allowed to guess at each step. |
| 156 | + - Simulated annealing. |
| 157 | + - There must exist a path in a random automata from a given input state to an optimal state. |
| 158 | +- $P \text{ vs } NP$ |
| 159 | + - $NP$ is the class of all search problems. Search problems must be verifiable in poly-time. |
| 160 | + - $P$ is the class of search problems (NP problems) that can further be **solved** in poly-time. |
| 161 | + - $P \subset NP$ |
| 162 | + - Is there truly a separation? Are all problems in NP also in P? |
| 163 | + |
| 164 | +![[Pasted image 20260324150706.png]] |
| 165 | + |
| 166 | +- if $P \ne NP$, what are the intractable problems? |
| 167 | + - These are the $NP$-complete problems. |
| 168 | + - These are the hardest problems in the class NP |
| 169 | + - if $P \ne NP$, then all NP-complete problems are not in $P$. |
| 170 | + - There may be other problems in NP which are currently not in P, but are not classified as NP-complete. The NP-complete problems are just the problems which are **guaranteed** to be in NP if $P \ne NP$. |
| 171 | + - If there exists an NP-complete problem which can be solved in polynomial time, then $P=NP$. |
| 172 | + - One example is SAT. Theoretically we can reduce all search problems to SAT. Then if we later prove that SAT is in P, then all search problems are in P. Thereby P=NP. |
| 173 | + |
| 174 | +## SAT is NP-Complete |
| 175 | +- "SAT is NP-complete" means |
| 176 | + 1. SAT $\in NP$. Refer to earlier for the proof that SAT $\in NP$ |
| 177 | + 2. SAT is the hardest problem in the class of NP problems. |
| 178 | + - What does this mean? It means that SAT is the "least likely" to have an "efficient" (poly time) solution. |
| 179 | + - If we can solve SAT in poly-time, then we can solve every problem in NP in poly-time. |
| 180 | + - For all of the problems in NP, there is a polynomial-time reduction to the SAT problem. |
| 181 | + - If then we have an efficient algorithm for SAT, then all NP problems can share that solution. |
| 182 | +- if $P \ne NP$ then $SAT \notin P$. |
| 183 | +- We can logically conclude that currently nobody knows whether $P=NP$ nor whether $SAT \in P$. |
| 184 | + - Prizes unclaimed. |
| 185 | + - Fields medals not-awarded |
| 186 | + |
| 187 | +## Reductions |
| 188 | +- Consider problems A and B |
| 189 | +- Reducing A to B |
| 190 | +- Example |
| 191 | + - A = Colorings |
| 192 | + - B = SAT |
| 193 | +- Reduction symbolizing |
| 194 | + - $A \rightarrow B$ - "A can be reduced to B" |
| 195 | + - $A \le B$ - "A is at most B" |
| 196 | + - "B" is the "harder" problem. |
| 197 | + - A solution for A doesn't necessarily solve B. |
| 198 | +- If we can solve problem B in poly-time, then we can use that algorithm to solve A in poly-time (as long as there exists a P-time algorithm to reduce A to B). |
| 199 | + |
| 200 | +### How to do a reduction? |
| 201 | +$Colorings \rightarrow SAT$ |
| 202 | +- Suppose that there is a poly-time algorithm alg for SAT, and we use it to get a poly-time alg. for colorings. |
| 203 | +- Put the SAT algorithm in a black box. We don't care how it works, just the input/output/runtime spec. |
| 204 | + - Use some transformation $f$ to transform an instance of B into an instance of A. |
| 205 | + - This produces either some solution to A, or "NO" |
| 206 | + - We use another transformation $h$ to transform the solution to A into a solution for B. |
| 207 | + - If A had no solution, then B has no solution. We don't need to transform "NO". |
| 208 | + |
| 209 | +![[Pasted image 20260324152336.png]] |
| 210 | + |
| 211 | +- $I_B$ - instance of problem B. |
| 212 | +- $I_A=f(I_B)$ - instance of problem A. |
| 213 | +- $S_A=Alg_A(I_A)$ - Solution to problem A |
| 214 | +- $S_B=h(S_A)$ - Solution to problem B |
| 215 | +- $S_B=h(Alg_A(f(I_B)))$ - The full chain of events. |
| 216 | +- If $S_A=NO$, then $h(S_A)=NO$ |
| 217 | + |
| 218 | +### More on Reductions |
| 219 | +- We need to define $f$ and $h$ |
| 220 | +- $f:$ input for colorings $\rightarrow$ input for SAT |
| 221 | + - Input for colorings: $G,k$ |
| 222 | + - Input for SAT: $f(G,k)$ |
| 223 | +- $h:$ solution for $f(G,k)$ $\rightarrow$ solution for colorings |
| 224 | + - Solution for SAT: $S$ |
| 225 | + - Solution for Colorings: $h(S)$ |
| 226 | +- Need to prove |
| 227 | + - if $S$ is a solution to $f$, then $h(S)$ is a solution to the original input. |
| 228 | + - $S$ is a solution to $f(G,k)$ $\iff$ $h(S)$ is a solution to $I=(G,k)$ |
| 229 | + |
| 230 | +See [[06.1 - NP - Theory Guidance]] for more info on what's required here. |
| 231 | + |
| 232 | +## NP-completeness Proof |
| 233 | +- To show: Independent Sets (IS) problem is NP-complete |
| 234 | +- We need to show: |
| 235 | + - $IS \in NP$. This is fairly straightforward in most cases. |
| 236 | + - For every problem $A \in NP$, we need to show $A \rightarrow IS$ |
| 237 | + - How? |
| 238 | + - In practice, do we transform the SAT problem into $IS$? |
| 239 | +- Suppose that we know SAT is NP-complete. |
| 240 | + - for all $A \in NP$, $A \rightarrow SAT$ |
| 241 | + - Suppose we show $SAT \rightarrow IS$ |
| 242 | + - If so, then for all $A \in NP$ there exists a double reduction $A \rightarrow SAT \rightarrow IS$ |
| 243 | + - We can shorten this chain to $$\forall_{A \in NP} \space A \rightarrow IS$$ |
| 244 | +- For a given problem B that _might_ be NP-complete, we simply need to show that there exists an NP-complete problem that can be reduced to B. SAT is the common one since it's so flexible, but there may be a different NP-C problem which is easier to reduce. |
| 245 | + |
| 246 | +## Practice Problems |
| 247 | +- [[8.1 - TSP optimization versus search (TODO)]] |
| 248 | +- [[8.2 - Search vs Decision (TODO)]] |
0 commit comments