The Federal Reserve’s effort to shrink the size of its balance sheet is moving into a more uncertain phase as a key proxy of excess liquidity has been very nearly extinguished.