Biological membranes consist of hundreds of different lipids that together with the embedded transmembrane (TM) proteins organize themselves into small nanodomains. In addition to this function of lipids, TM regions of proteins bind to lipids in a very specific manner, but the function of these TM region-lipid interactions is mostly unknown. In this review, we focus on the role of plasma membrane cholesterol, which directly binds to the αβ T cell antigen receptor (TCR), and has at least two opposing functions in αβ TCR activation. On the one hand, cholesterol binding to the TM domain of the TCRβ subunit keeps the TCR in an inactive, non-signaling conformation by stabilizing this conformation. This assures that the αβ T cell remains quiescent in the absence of antigenic peptide-MHC (the TCR's ligand) and decreases the sensitivity of the T cell toward stimulation. On the other hand, cholesterol binding to TCRβ leads to an increased formation of TCR nanoclusters, increasing the avidity of the TCRs toward the antigen, thus increasing the sensitivity of the αβ T cell. In mouse models, pharmacological increase of the cholesterol concentration in T cells caused an increase in TCR clustering, and thereby enhanced anti-tumor responses. In contrast, the γδ TCR does not bind to cholesterol and might be regulated in a different manner. The goal of this review is to put these seemingly controversial findings on the impact of cholesterol on the αβ TCR into perspective.