So the only time your vote actually affects anything is when your vote decides an election. If your vote does not decide an election, it wouldn't have turned out differently if you haven't voted, therefore, your vote didn't made a difference.
So to increase the probability that your vote sways an election, you and a bunch of people get together and hold a private voting before the real election. Everyone in your group votes for whoever wins the private voting (and no one cheats and votes for a loser because that's who they want to win), like how most US states will cast all their electoral ballots for whichever candidate wins their state. Because the only way your vote matters is when your vote sways an election, the bigger the pool of voters in your private election, the more likely it is your block of voters sways an election.
Since California is the biggest voting block, don't they actually have the most ability to sway an election (if you assume all the other states are voting randomly)?
Similar to imagining a group of 3 people. Person A has 6 votes. Person B has 4 votes. Person C has 3 votes. They all have the same amount of voting power because the only way for something to get a majority of votes is to have any 2 people vote for it. It's not votes that matter, but your ability to sway on your own.