Enabling MMU and CACHES lead to "SDHC: Error: No Card Inserted"

Hello,
I'm trying to enable MMU and CACHES in SAMA5D2-D1G based board but I got

AT91Bootstrap 4.0.6-dirty (2023-05-05 12:39:18)

SD/MMC: Image: Read file u-boot.bin to 0x23f00000
SDHC: Error: No Card Inserted
*** FATFS: f_open, filename: [u-boot.bin]: error
SD/MMC: Failed to load image

I let CONFIG_MMU_TABLE_BASE_ADDR = 0x20000000 as default.
Any suggestions? 
