Hey Natasha,
In short, you look at basicity to determine nucleophilicity when you are comparing atoms in the same row (ie O/N), and you look at size when comparing atoms in a column (ie S/O). The halogens follow this rule as well, but are more complicated due to solvent (see below).
A more lengthy and complete explanation of nucleophilicity trends:
There are a lot of conflicting contextual rules, I will try to put out a set here that reconciles this, framed according to periodic trends.
1. Across a row, nucleophilicity decreases with electronegativity, as more electronegative elements want to give away electrons less. (This is the same as saying nucleophilicity increases with basicity, as more electronegative elements are less basic).
Rule 1 assumes that your atoms are in the same ionization state.
For example, we would predict from this trend that Na would be the most nucleophilic of its row. In fact, it is so nucleophilic that you only tend to find it as Na+ in Ochem reactions. Because Na+ has already given its electrons away, it is not nucleophilic at all.
In short, you have to compare neutral atoms with neutral atoms, you can't compare OH- to NH3 as easily in this case.
2. Down a column, we no longer consider acidity/basicity as atom size is more important than electonegativity. We just consider if the atom is smaller or larger and ignore everything else.
The bigger an atom is, the more likely it will bump into an electrophile at the right orientation, resulting in a reaction. This is for example why S is more nucleophilic than O.
We usually consider these two rules for the common atoms in biology, for example N, C, O, S, P, and a couple extra elements. These are the rules I would want to have down for the MCAT
Another quick note about 1 and 2 is we usually only use one rule at a time - note in review question on page 163 we compare N C and O at the same ionization state (-1). If you had to compare N and S you would have to weigh the importance of rule 1 and 2, which isn't really possible to do except by experiment and is therefore outside of the scope of things you would be expected to know for the MCAT
3. Halides are complicated exceptions and solvent dependant. This is more complicated and less useful for the MCAT than rules 1 and 2
Most nucleophiles you look at don't have a full blown negative charge on them. For F, Cl, Br and I they will bear this full blown negative as nucleophiles, so how nucleophilic they are depends on how well they stabilize negative charge. For the cases of negative charge, because they are all so far on the right we consider their electronegativity about the same and consider size more significant.
Because F- stabilizes negative charge the least, being smallest, it is the most intrinsically reactive, and therefore nucleophilic. However, it is also the most basic. That means in a polar protic solvent, it will hydrogen bond a lot, preventing it from acting as a nucleophile.
What I am trying to do with the framework is tell you when to use one of the conflicting rules, just as in periodic trends.
This is a lot to digest, I hope this framework helps. On the bright side, if you really absorb this framework, identifying nucleophiles gets a lot easier. I would try to use this framework and practice it on examples instead of memorizing.
Please let me know if you have any questions.
Katt