Federal Reserve Bank of Philadelphia President Patrick Harker said higher interest rates are making it more challenging for first-time home buyers by raising borrowing costs and limiting inventory, which is also leading to higher home prices