Eigenvalues describe the directions along which a linear transformation stretches or compresses space. Behind every PCA dimension, every stability analysis, and every spectral decomposition lies the same question: What scalars \(\lambda\) satisfy \(A\mathbf{v} = \lambda \mathbf{v}\) for some non-zero vector \(\mathbf{v}\)? Here we clarify the polynomial road map to those scalars and show how it generalizes from 2×2 matrices to higher dimensions, including why principal minors appear in the coefficients.

The Characteristic Equation

For any square matrix \(A\), the eigenvalues are the roots of the characteristic polynomial:

2×2 refresher

When \(A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}\), the determinant expands to

Hence the trace \((a+d)\) appears as the \(\lambda\)-coefficient and the determinant \(ad-bc\) is the constant term. Solving the quadratic gives the eigenvalues.

Worked Examples: \(n=3\) and \(n=4\)

\(3 × 3\)

Let

Then \(A - \lambda I\) becomes

For a 3×3 matrix \( A = \begin{bmatrix} a & b & c \\ d & e & f \\ g & h & i \end{bmatrix} \), the characteristic polynomial expands as

where \(\sigma_2(A)\) is the sum of the 2×2 principal minors:

Principal minors are the determinants of every square submatrix formed by selecting the same rows and columns. They appear because the expansion of \(\det(A - \lambda I)\) naturally produces sums over these subdeterminants—each coefficient gathers the contributions of principal minors of increasingly large size, which mathematically captures the pairwise, triple, etc., interactions between eigenvalues.

Notice:

• The \( \lambda^3 \) coefficient is 1 because the polynomial is monic.
• The \(-\text{tr}(A)\) term mirrors the 2×2 case.
• The constant term is \(-\det(A)\).
• Intermediate coefficients aggregate principal minors.

Numeric example. For

the trace is 11, \(\sigma_2(A) = (1\cdot4 - 0\cdot2) + (1\cdot6 - 0\cdot3) + (4\cdot6 - 0\cdot5) = 4 + 6 + 24 = 34\), and the determinant is \(1\cdot4\cdot6 = 24\). The characteristic polynomial becomes \(\lambda^3 - 11\lambda^2 + 34\lambda - 24\), whose roots are the eigenvalues of \(A\).

\(4 × 4\)

For a 4×4 matrix, the pattern continues. Without loss of generality, \(A\) produces

where:

• \(\sigma_2(A)\) sums all 2×2 principal minors,
• \(\sigma_3(A)\) sums all 3×3 principal minors,
• Signs alternate as dictated by the expansion of \(\det(A - \lambda I)\),
• Trace and determinant remain the first and last invariants, and the intermediate \(\sigma_k\) tie into symmetric polynomials of eigenvalues.

Each \(\sigma_k\) aggregates the determinants of the \(k \times k\) principal submatrices because the Leibniz expansion of the determinant iterates over all permutations of row/column pairs. Keeping track of these minors lets us express the characteristic polynomial coefficients without expanding every term manually, which is why they are a compact way to encode the invariant relationships between eigenvalues and the matrix entries.

Numeric example. Take

The trace is 14, \(\sigma_2(A)\) collects the 2×2 minors (e.g., \(2\cdot3, 2\cdot4, 3\cdot4\) plus the shifts introduced by the 1s), and the determinant is \(2\cdot3\cdot4\cdot5 = 120\). The characteristic polynomial becomes \(\lambda^4 - 14\lambda^3 + \sigma_2(A)\lambda^2 - \sigma_3(A)\lambda + 120\), encoding the same invariants as before but for four eigenvalues.

These expressions remind us that the characteristic polynomial encodes global constraints: eigenvalues sum to the trace, pairwise products sum to \(\sigma_2(A)\), triple products to \(\sigma_3(A)\), and so on. Even though solving quartics or higher-degree polynomials analytically becomes impractical, numerical methods target the same invariants, so the conceptual road map stays intact.

Method 1: Manual Calculation (Educational)

 1import math
 2
 3def calculate_eigenvalues_2x2(A: list[list[float | int]]) -> list[float]:
 4    if len(A) != 2 or len(A[0]) != 2 or len(A[1]) != 2:
 5        raise ValueError("Input must be a 2x2 matrix.")
 6
 7    trace = A[0][0] + A[1][1]
 8    determinant = A[0][0] * A[1][1] - A[0][1] * A[1][0]
 9
10    discriminant = trace**2 - 4 * determinant
11    if discriminant < 0:
12        raise ValueError("Matrix has complex eigenvalues in this implementation.")
13
14    eigenvalue_1 = (trace + math.sqrt(discriminant)) / 2
15    eigenvalue_2 = (trace - math.sqrt(discriminant)) / 2
16
17    return [eigenvalue_1, eigenvalue_2]

This hands-on route is excellent for reinforcing how the trace and determinant become coefficients of the characteristic polynomial and how complex eigenvalues arise when the discriminant is negative.

Method 2: NumPy for the General Case

 1import numpy as np
 2
 3def calculate_eigenvalues_numpy(matrix: list[list[float | int]]) -> list[complex]:
 4    A = np.array(matrix)
 5
 6    if A.shape[0] != A.shape[1]:
 7        raise ValueError("Input must be a square matrix.")
 8
 9    eigenvalues = np.linalg.eigvals(A)
10    return eigenvalues.tolist()

NumPy hides the polynomial expansion but still solves \(\det(A - \lambda I) = 0\). It handles arbitrary square matrices, complex eigenvalues, and higher dimensions effortlessly.

Conclusion

Whether you expand a characteristic polynomial by hand for \(n=2\) or rely on NumPy for \(n>2\), the goal is the same: find the roots of \(\det(A - \lambda I)\). The trace and determinant continue to anchor the polynomial’s coefficients, while the intermediate terms encode sums of minors. Understanding this structure helps you make sense of the numerical results produced for large matrices and gives intuition for the algebra that underlies eigenvalue computation.

Intuition

The problem asking us to partition the given set of numbers into two sets such that the sum of the numbers in the two sets are equal.

For the sum of two sets to be equal the total sum of the numbers must be even. That means sum(numbers) % 2 == 0. by the forumla for the sum of the first n natural numbers, sum(numbers) = n * (n + 1) / 2. So that means sum of each set must be n * (n + 1) / 4.

Hence we got that n mod 4 is 0 or (n+1) mod 4 is 0, combining both we get n mod 4 must be 0 or 3 to be able to partition the numbers into two sets. So this becomes a necessary condition for the problem to be solvable.

but this is not a sufficient condition.

When n mod 4 = 0: Total sum is even, partition is possible. The set can be split so both subsets have the same number of elements (equal size).

Example: n=8

Pairs: (8,1), (7,2), (6,3), (5,4)

Assign two pairs to one set, the other two pairs to the other set—both sum to 18.

When n mod 4 = 3: Total sum is even, partition is possible.

The sets will have different sizes (one will have one more element). Example: n=7

Pairs: (7,1,6) sums to 14, rest (2,3,4,5) sum to 14.

The sum of each smallest–largest pair is always n+1.

For n mod 4 = 0: you can split pairs alternately between the two sets.

For n mod 4 = 3: this is a tricky case Consider some integer k such that we have n = 4k+3. that means n mod 4 = 3. Now sum of n numbers is n * (n + 1) / 2 = (4k+3) * (4k+4) / 2 = (4k+3) * (2k+2)

Mathematical Derivation

For n = 4k + 3, we assign:

• Set 1: numbers where i ≡ 1 or 2 (mod 4)
• Set 2: numbers where i ≡ 0 or 3 (mod 4)

Count of numbers in each residue class:

• ≡ 0 mod 4: k numbers (4, 8, …, 4k)
• ≡ 1 mod 4: k+1 numbers (1, 5, …, 4k+1)
• ≡ 2 mod 4: k+1 numbers (2, 6, …, 4k+2)
• ≡ 3 mod 4: k+1 numbers (3, 7, …, 4k+3)

Sum of each residue class:

≡ 0 mod 4:

4 + 8 + ... + 4k = 4(1 + 2 + ... + k) = 4 × k(k+1)/2 = 2k(k+1)

≡ 1 mod 4:

1 + 5 + 9 + ... + (4k+1)

Arithmetic series: first term = 1, last term = 4k+1, common difference = 4, terms = k+1

Sum = (k+1) × (1 + 4k+1)/2 = (k+1) × (4k+2)/2 = (k+1) × 2(k+1) = 2(k+1)²

≡ 2 mod 4:

2 + 6 + 10 + ... + (4k+2)

Arithmetic series: first term = 2, last term = 4k+2, common difference = 4, terms = k+1

Sum = (k+1) × (2 + 4k+2)/2 = (k+1) × (4k+4)/2 = (k+1) × 2(k+2) = 2(k+1)(k+2)

≡ 3 mod 4:

3 + 7 + 11 + ... + (4k+3)

Arithmetic series: first term = 3, last term = 4k+3, common difference = 4, terms = k+1

Sum = (k+1) × (3 + 4k+3)/2 = (k+1) × (4k+6)/2 = (k+1) × 2(k+3) = 2(k+1)(k+3)

Total sum verification:

Total sum = 2k(k+1) + 2(k+1)² + 2(k+1)(k+2) + 2(k+1)(k+3)
         = 2(k+1) × [k + (k+1) + (k+2) + (k+3)]
         = 2(k+1) × (4k + 6)
         = 2(k+1) × 2(2k + 3)
         = 4(k+1)(2k + 3)

Each set should sum to: 2(k+1)(2k + 3)

Sum of each set:

Set 1 (≡1 or ≡2 mod 4):

Sum = 2(k+1)² + 2(k+1)(k+2) = 2(k+1) × [(k+1) + (k+2)] = 2(k+1) × (2k + 3)

Set 2 (≡0 or ≡3 mod 4):

Sum = 2k(k+1) + 2(k+1)(k+3) = 2(k+1) × [k + (k+3)] = 2(k+1) × (2k + 3)

Both sets sum to 2(k+1)(2k + 3), which is exactly half the total sum.

Solution

 1def main():
 2    n = int(input())
 3    if n % 4 not in (0, 3):
 4        print("NO")
 5        return
 6    
 7    set1 = []
 8    set2 = []
 9    toggle = True
10    
11    if n % 4 == 0:
12        low = 1
13        high = n
14        while low <= high:
15            if toggle:
16                set1.append(low)
17                set1.append(high)
18            else:
19                set2.append(low)
20                set2.append(high)
21            low += 1
22            high -= 1
23            toggle = not toggle
24    else:
25        for i in range(1, n+1):
26            if i % 4 == 1 or i % 4 == 2: # 
27                set1.append(i)
28            else:
29                set2.append(i)
30    
31    # Output the result
32    print("YES")
33    print(len(set1))
34    print(" ".join(map(str, set1)))
35    print(len(set2))
36    print(" ".join(map(str, set2)))
37
38if __name__ == "__main__":
39    main()

Intuition

This problem wants us to find the missing number in a sequence of numbers from 1 to n. The naive approach is to use a hash map to store the numbers and then find the missing number. But this is not efficient.

The idea is to use the formula for the sum of an arithmetic series. The sum of an arithmetic series is given by the formula:

where n is the number of terms, a is the first term, and l is the last term. In this problem, the first term is 1 and the last term is n. So the sum of the series is given by the formula:

now if we accumulate the sum of the numbers in the list, we can find the missing number by subtracting the sum of the numbers in the list from the sum of the series. Since only one number is missing, the sum of the numbers in the list will be less than the sum of the series and the difference will be the missing number.

Solution

 1def main():
 2    n = int(input())
 3    nums_list = input().strip()
 4    nums = nums_list.split()
 5    total = 0
 6    for num in nums:
 7        total += int(num)
 8    tot = n * (n + 1) // 2
 9    print(tot - total)
10
11
12if __name__ == "__main__":
13    main()

Some formulae for arithmetic series

nth term

where a is the first term, d is the common difference, and n is the number of terms.

sum of the first n terms

where n is the number of terms, a is the first term, and l is the last term.

sum of the first n terms

where n is the number of terms, a is the first term, and d is the common difference.

Intuition

This problem is a direct implementation of the algorithm described in the problem statement.

Given a number n, we need to apply the following algorithm:

1. If n is even, divide it by 2.
2. If n is odd, multiply it by 3 and add 1.
3. Repeat the process until n becomes 1.

Solution

1def weird_algorithm(n):
2    while n != 1:
3        print(n, end=" ")
4        if n % 2 == 0:
5            n = n // 2
6        else:
7            n = 3 * n + 1
8    print(n)

Geek fact

This is a famous problem in the field math called the Collatz conjecture.

Collatz conjecture is a conjecture in mathematics that states that for any positive integer n, the sequence of numbers generated by the following algorithm will eventually reach 1.

So far, the conjecture has been verified for all positive integers up to 2^68.

Wikipedia link: Collatz conjecture

I recently dove deep into how LLMs like GPT-3, GPT-4, and open-source models generate text, and I was fascinated by the sheer number of knobs and dials available to control their behavior. These aren’t just abstract concepts,they’re the tools that make the difference between getting a coherent essay or a rambling mess, between creative poetry or repetitive drivel.

In this post, I’ll break down the key generation-time hyperparameters with practical examples, mathematical foundations, and real-world implications. We’ll go beyond the surface-level explanations and understand why each parameter matters and how to tune them effectively.

Temperature: The Creativity Knob

Temperature is arguably the most important parameter you’ll encounter when using LLMs. It’s like adjusting the “personality” of the model.

What it does: Temperature scales the logits (raw prediction scores) before they’re converted to probabilities via softmax. Mathematically:

where z_i is the logit for token i, and T is temperature.

Why it works: At lower temperatures (T < 1), the probability distribution becomes sharper, high-probability tokens become even more likely, leading to more predictable, conservative outputs. At higher temperatures (T > 1), the distribution flattens, giving low-probability tokens a better chance.

Real-world example: When generating code, you want T=0.1 for deterministic, correct syntax. For creative writing, T=0.8-1.2 gives you that spark of originality. I’ve seen T=2.0 produce genuinely surprising and creative outputs, though sometimes bordering on incoherent.

Typical range: 0.0 (completely deterministic) to 2.0 (highly random). Most applications use 0.7-1.0.

Top-k Sampling: Quality Control

Top-k is like telling the model, “Only consider your top k most confident predictions.”

What it does: After computing all token probabilities, sort them and keep only the top k tokens, then renormalize their probabilities.

Why it works: It prevents the model from picking extremely unlikely tokens that might derail the generation. For example, if the model is writing about “coffee,” top-k=50 ensures it doesn’t suddenly switch to discussing quantum physics unless that was already in its top 50 predictions.

Practical tip: For factual writing, use lower k (10-40) to stay on topic. For creative tasks, higher k (50+) allows more flexibility. I’ve found k=1 essentially gives you greedy decoding,always pick the most likely token.

Top-p (Nucleus Sampling): Dynamic Quality Control

Top-p takes top-k’s idea but makes it adaptive.

What it does: Instead of a fixed number of tokens, select the smallest set whose cumulative probability exceeds p. So if p=0.9, it might use 10 tokens one time and 50 tokens another time, depending on the probability distribution.

Why it works: In scenarios where the model is very confident (sharp distribution), it uses fewer tokens. When uncertain (flat distribution), it considers more options. This is particularly useful for maintaining coherence while allowing creativity.

Example: Writing a technical explanation,early tokens are predictable, so top-p might use 5-10 tokens. When getting creative, it expands to 20-30 tokens. I’ve seen this outperform top-k in many creative writing tasks because it adapts to the model’s confidence level.

Typical range: 0.8-1.0. Values below 0.5 can be too restrictive.

Logit Bias: Manual Steering

This is the parameter for when you want to force the model’s hand.

What it does: Directly add or subtract values from specific token logits before sampling:

Why it works: Want to ban certain words? Set their bias to -100. Want to encourage specific terminology? Add positive bias. I once used this to prevent an LLM from using certain brand names in generated content.

Practical applications: Content moderation, style enforcement, or domain-specific vocabulary control. Particularly useful in enterprise settings where you need to avoid certain terms.

Repetition Penalty: Fighting the Echo Chamber

Ever noticed how LLMs can get stuck repeating phrases? This parameter fights that.

What it does: For tokens that have appeared recently, divide their logits by a penalty factor r > 1:

Why it works: It makes previously used tokens less attractive, encouraging the model to explore new vocabulary. Without this, models can get into loops like “The best coffee is the best coffee is the best coffee…”

Real-world use: Essential for long-form content generation. I use values around 1.1-1.2 for most tasks, but for very creative work, sometimes 1.05 or lower to allow some intentional repetition for emphasis.

Stop Sequences: The Off Switch

Sometimes you need the model to know when to stop talking.

What it does: Define specific strings that, when generated, halt the generation process.

Why it works: Prevents runaway generation or ensures structured output. For example, when generating code, you might stop at the next function definition.

Examples: "\n\n", "END", "</response>". I once used this to generate structured JSON by setting the stop sequence to "}".

Max Tokens: Setting Boundaries

The word limit for your AI writer.

What it does: Caps the total number of tokens generated in a single response.

Why it works: Prevents excessive output and manages costs/compute time. In API contexts, this also affects billing.

Considerations: Different models have different limits (GPT-4: 4096, Claude: 8192). Always set this lower than the model’s maximum to leave headroom.

Frequency Penalty: Discouraging Word Hoarders

Similar to repetition penalty, but based on overall frequency in the generated text.

What it does: Penalize tokens based on how often they’ve appeared:

Why it works: Prevents overuse of common words. If “the” appears 50 times, it gets increasingly penalized.

Use cases: Particularly effective for generating diverse content or when you want to avoid repetitive language patterns.

Presence Penalty: Topic Freshness

This encourages introducing new concepts rather than revisiting old ones.

What it does: Penalize tokens that have appeared at all in the current generation:

where presence(i) is 1 if the token appeared, 0 otherwise.

Why it works: Promotes topic exploration. Without it, models can get stuck discussing the same ideas repeatedly.

Difference from frequency penalty: Frequency counts occurrences, presence is binary. Use presence penalty for encouraging breadth of topics.

Num Keep: Context Continuity

For long conversations or documents, this maintains coherence.

What it does: When refreshing the context window, retain the last N tokens from the previous context.

Why it works: Prevents abrupt topic changes in long-form generation. Imagine writing a novel,the model needs to remember what happened in the previous chapter.

Typical range: 100-500 tokens, depending on the task complexity.

Seed: Reproducibility Control

For when you need consistent outputs.

What it does: Initializes the random number generator with a fixed value, ensuring identical outputs for identical inputs.

Why it works: Crucial for testing, debugging, and applications requiring deterministic behavior. Without seeding, the same prompt can produce different outputs.

Implementation note: Not all APIs expose seed control, but when available, it’s invaluable for quality assurance.

Repeat Last N: Short-Term Memory

A more targeted repetition control.

What it does: Prevents tokens from the most recent N tokens from being selected.

Why it works: Stops immediate repetition like “I like coffee. I like coffee. I like coffee.”

Typical range: 1-10 tokens. Higher values can prevent natural repetition that’s actually desirable.

Mirostat: Adaptive Temperature

This is for the advanced users who want the model to maintain consistent “surprise” levels.

What it does: Dynamically adjusts temperature to target a specific perplexity level using parameters τ (target perplexity) and η (adjustment rate).

Why it works: Maintains consistent creativity levels throughout generation. Traditional temperature can lead to increasingly erratic outputs as generation progresses.

Typical range: τ: 20-100, η: 0.05-0.2. Requires experimentation to find good values for your use case.

Putting It All Together

These parameters don’t exist in isolation, they interact in complex ways. For most applications, you’ll primarily tune temperature, top-p, and repetition penalties. But understanding all of them gives you the full toolkit for controlling LLM behavior.

Remember: there’s no “perfect” set of parameters. The best settings depend on your specific use case, model, and desired output characteristics. Experimentation and iteration are key.

What parameter have you found most useful in your LLM work? I’d love to hear your thoughts!

ʕ •ᴥ•ʔ

1. Question 1 From deep-ml.com Essentially a dot product of a matrix and a vector an happen only if the number of columns in the matrix is equal to the number of elements in the vector.

That is if A[m,n] and B[n], then A*B is defined.

So in this case, the matrix is 2x3 and the vector is 3x1, so the dot product is defined.

The result is a 2x1 matrix.

The result is:

Solution

 1def matrix_dot_vector(a: list[list[int|float]], b: list[int|float])
 2 -> list[int|float]:
 3	# Return a list where each element is the dot product of a row of 'a' with 'b'.
 4	# If the number of columns in 'a' does not match the length of 'b', return -1.
 5    if len(a[0]) != len(b):
 6        return -1
 7    c = []
 8    for i in range(len(a)):
 9        sum = 0
10        for j in range(len(a)):
11            sum += a[i][j]*b[j]
12
13        c.append(sum)
14	return c