A pearl millet inbred germplasm association panel (PMiGAP) comprising 250 inbred lines, representative of cultivated germplasm from Africa and Asia, elite improved open-pollinated cultivars, hybrid parental inbreds and inbred mapping population parents, was recently established. This study presents the first report of genetic diversity in PMiGAP and its exploitation for association mapping of drought tolerance traits. For diversity and genetic structure analysis, PMiGAP was genotyped with 37 SSR and CISP markers representing all seven linkage groups. For association analysis, it was phenotyped for yield and yield components and morpho-physiological traits under both well-watered and drought conditions, and genotyped with SNPs and InDels from seventeen genes underlying a major validated drought tolerance (DT) QTL. The average gene diversity in PMiGAP was 0.54. The STRUCTURE analysis revealed six subpopulations within PMiGAP. Significant associations were obtained for 22 SNPs and 3 InDels from 13 genes under different treatments. Seven SNPs associations from 5 genes were common under irrigated and one of the drought stress treatments. Most significantly, an important SNP in putative acetyl CoA carboxylase gene showed constitutive association with grain yield, grain harvest index and panicle yield under all treatments. An InDel in putative chlorophyll a/b binding protein gene was significantly associated with both stay-green and grain yield traits under drought stress. This can be used as a functional marker for selecting high yielding genotypes with ‘stay green’ phenotype under drought stress. The present study identified useful marker-trait associations of important agronomics traits under irrigated and drought stress conditions with genes underlying a major validated DT-QTL in pearl millet. Results suggest that PMiGAP is a useful panel for association mapping. Expression patterns of genes also shed light on some physiological mechanisms underlying pearl millet drought tolerance.