The study was done in a mixed oak forest located between San Juan Tlacotenco (Tepoztlán) and Coajomulco (Huitzilac), Morelos, Mexico in the Tepozteco national park (99°02’00”–99°12’55”N, 18°53’20”–19°05’30”W) (Cruz-Fernández et al. 2011; Suppl. material 1). The temperature and annual rain averages are 15.9°C and 1478 mm (mainly falling from July to September), respectively (Comision Nacional del Agua unpubl. data; Suppl. material 1). The mixed oak forest reaches 25–30 m in height and is composed of at least 30 tree species (Diameter at Breast Height = DBH > 10 cm); the most frequent species are Quercus rugosa Née (39.3% of the individuals; Fagaceae), Q. castanea Née (16.4%), Ternstroemia lineata DC. (11.9%; Pentaphylacaceae), Q. obtusata Bonpl. (5%), Garrya laurifolia Benth. (3.9%; Garryaceae), and Arbutus xalapensis Kunth (3.5%; Ericaceae). Altogether, these species make up 80% of the tree forest individuals (DBH > 10 cm) (Cruz-Fernández et al. 2011).

In August–September 2013 (rainy season), we randomly selected ten sampling points within the mixed oak forest (Suppl. material 1). To do so, we drove the road from Coajomulco to San Tlacotenco and from this town toward the Mexico-Tepoztlán highway. This road originated from what was previously a railway track and runs through the forested area. With the help of ten random numbers, we selected ten positions along the road, and each point was marked (flagging tape). From each point marked position on the road, we advanced 50 m into the forest to establish a sampling 10 × 100 m transect.

In each transect, we searched for CWD (diameter ≥ 10 cm) lying on the forest floor (i.e. fallen branches, trunks, and stumps). We measured the larger diameter at one of its extremes for each CWD, its length, plant coverage (mosses, fungi, and each vascular plant species), and the number of orchid individuals. To estimate coverages, we measured the length of each fungus/plant species on the CWD and divided it by the CWD length. Except for orchids (identified in the field with a field guide; Espejo-Serna et al. 2002), plants were collected and identified at the herbarium HUMO (Universidad Autonoma del Estado de Morelos), but most were small plants without reproductive structures. We used the Vanderwel et al. (2006) scale for each CWD to estimate its decay status. This scale has five decay classes, going from recently fallen branches and trunks (i.e. intact wood and bark) to completely decayed wood resembling soil (Suppl. materials 2, 3).

We experimentally tested seed germination in forest soil, bark, and decayed CWD. In this experiment, we used seeds of two of the most common orchids in the study area: Malaxis brachyrrhynchos and, in the absence of true epiphytic Malaxidinae, Stelis retusa (Espejo-Serna et al. 2002). Both species belong to different Epidendroidae tribes and share an epiphytic common ancestor (Zhang et al. 2023). In our study area, they co-occur in the same forest and have had similar opportunities to colonize the same substrates, enabling tests of whether germination differences drive substrate preferences. Malaxis brachyrrhynchos is terrestrial, and Stelis retusa is a true epiphyte, and both have restricted distribution in Morelos (Espejo-Serna et al. 2002; Cruz-Fernández et al. 2011; Mena-Jiménez et al. 2024). To ensure fruit production, 10 plants of each species were manually cross-pollinated during the rainy season of 2019 (June–October) and monitored until fruit maturity (when fruits open spontaneously). Cross-pollination was performed by moving pollen between clearly spaced plants. We obtained 44 fruits of Malaxis brachyrrhynchos (1.422 g of seeds) and 13 fruits of Stelis retusa (0.367 g of seeds). For each species, we estimated the number of seeds per gram following the method of Emeterio-Lara et al. (2018). In M. brachyrrhynchos, the estimated average number of seeds with embryo was 599259 ± 123775 (mean ± SD, n = 10; coefficient of variation = 20.65%), and the number of seeds without embryo was 54815 ± 24741 (coefficient of variation = 45.13%); while in S. retusa there were 638519 ± 65495 (10.26%) and 185185 ± 47108 (25.43%) seeds with and without embryo, respectively.

In May 2020, we split the sample of seeds into 105 sets of 0.003 mg (containing ca 1798 seeds of M. brachyrrhynchos and ca 1916 seeds of S. retusa). Each seed set was put inside a 4 × 3 cm mesh envelope (Nitex screen, NTX50, 50 μm opening). With this method, it is possible to study the germination of dust seeds, allowing them to interact with microorganisms without seed losses (Rasmussen and Whigham 1993, 1998). We randomly assigned each seed set to one of seven treatments for each orchid species (15 seed sets per treatment) (Suppl. material 4). A treatment is a specific, distinct condition formed by combining levels of one or more independent factors (Montgomery 2019). The treatments were: forest soil (soil), decayed CWD (CWD), Quercus rugosa bark (bark), and four substrate combinations (1:1 soil-CWD, 1:1 soil-bark, 1:1 CWD-bark, and 1:1:1 soil-CWD-bark).

Three CWD in decay class 4 (Suppl. materials 2, 3) were sampled for soft, decayed coarse woody debris collection. Decayed wood was collected from the inner parts of CWD to avoid soil. Five upper soil subsamples (10 cm depth, avoiding coarse litter) were collected 5 m from the CWD and pooled. Bark samples were collected from the trunks of four Quercus rugosa with a bark scraper (Lightweight curved bark scraper, 250 mm, OX 370-2500, Forestry Suppliers); this tree species is dominant in some areas of the forest and hosts S. retusa (Mena-Jiménez et al. 2024). During the bark collection process, we scraped the outer bark layer and collected small pieces, avoiding sampling large, deep pieces that could expose the tree’s inner tissue. To prevent bark from mixing with forest soil, the forest soil floor was covered with plastic. Each substrate was combined into a single sample and manually pulverized when necessary. When necessary, bark was lightly homogenized by hand before being used in the experiments.

Because of the closure of activities caused by the COVID-19 pandemic, we could not conduct the experiment under field conditions or in a greenhouse at the Universidad Autonoma del Estado de Morelos. Therefore, we performed the seed germination experiment ex situ in the garden of the first author (M.V. S.-M.). Ex situ experiments have several strengths because they allow more accurate control of secondary factors (e.g. site variability) while manipulating the factors (e.g. substrate type) (Brundrett et al. 2003). Each seed set was sown in a disposable plastic gelatine single mould (transparent, 50 ml), each mould was ¾ filled with a substrate, and the envelope with the seeds was buried lightly (3–5 mm). The substrate was moistened every day (8:30–10:30 h). Five seed sets were reviewed with an 8× magnifier every two weeks to look for seed germination. When the first germination was observed, all the seed envelopes were examined. We opened each seed envelope and counted the number of germinated and non-germinated seeds under a microscope. Germination is the process by which the dehydrated, resting embryo activates, beginning with the imbibition of the seed and ending with the protrusion of the radicle (Bewley et al. 2013). However, when an orchid seed germinates, it does not develop a radicle (Dressler 1981), germination is usually taken as the rupture of the seed coat by the embryo (Rasmussen and Whigham 1998; Brundrett et al. 2003; Tsutsumi et al. 2011; Khamchatra et al. 2016; Emeterio-Lara et al. 2018; Zhang et al. 2022; Rammitsu et al. 2023), once the orchid seed coat breaks (because of embryo growth) the seed loses its capacity to disperse and ceases to be a seed. After seed germination (known as stage 1 or 2) (e.g. Khamchatra et al. 2016), several developmental stages of the seedling can be recognized (Arditti 1967; Dressler 1981).

Data analyses were done in R v.4.2.1 (R Core Team 2022), using the ggplot2 v.3.5.0 (Wickham 2016) for graphics, magrittr v.2.0.3 (Bache and Wickham 2022) and Summarytools v.1.0.1 (Comtois 2022) for descriptive statistics, and those cited further.

To test whether the abundance of Malaxis was related to CWD diameter, decay class, species richness (S), and fungal cover (%), we compared the performance of 16 candidate models, including all combinations of these variables. All predictor variables were standardized before analysis and were not strongly correlated with one another (Suppl. material 5). Additional variables were excluded because their inclusion substantially increased model overdispersion (> 7) or because they contained excessive zeros (e.g. cover of all plant species). The 16 generalized linear models were built for a Poisson response variable (i.e. the number of Malaxis individuals per CWD). Model comparisons were based on the quasi-AIC (QAICc) because of overdispersion (Pearson c_hat = 3.8) (Crawley 1993). Model selection was done with the library AICcmodavg (Mazerolle 2023). The model with the lowest quasi-AIC was taken as the best, and the rest differed from this if the change in quasi-AIC was > 2 (ΔQAICc) (Anderson 2008).

Seed germination of each orchid species was analysed with a generalized linear model for a binomial response variable (Crawley 1993). Following our full factorial experimental design, we tested each substrate’s effect (the main effects: bark, CWD, and soil) and their interactions in seed germination (i.e. the two- and three-way interactions of the main factors) (Crawley 1993; Zar 2010). Each model’s response variable was binomial (i.e. each seed germinated or not), and each model’s link function was logit (Crawley 1993). To fulfil the experimental design, the treatment with all substrates absent (none) was substituted for each species with a set of data on general mean germination (Crawley 1993; Zar 2010); in this way, we could decompose the deviance between all substrates and their interactions. We calculated the 95% confidence intervals for a binomial variable following Zar (2010) for each germination mean. When necessary, multiple comparisons between germination means were made using the R package multcomp v.1.4-26 (Bretz et al. 2011).

In the ten transects, we counted 89 pieces of CWD (eight stumps and 81 fallen branches and trunks) on the forest floor. The mean length of CWD was 393 cm (± 349 cm, minimum = 12 cm, maximum = 1356 cm) with a mean diameter of 24.8 cm (± 11.8 cm, minimum = 10 cm, maximum = 57 cm) (Suppl. material 6). The diameter distribution was skewed to the left, and large CWD were infrequent (Fig. 1).

The average number of CWD per 0.1 ha transect was 8.9 (± 4.6 CWD/0.1 ha, minimum = 3, maximum = 19). The availability of CWD in the forest is low. The average area of the forest floor covered by CWD is 9.1 ± 5.8 m²/0.1 ha (minimum = 1.5 m²/0.1 ha, maximum = 19.0 m²/0.1 ha), meaning that the average percentage of forest floor covered by CWD is 0.009 ± 0.006% (minimum = 0.002%, maximum = 0.02%).

Most CWD were in decay class 2 (36.0%) (Table 1), and 9.0% were in decay class 1. With one exception, all CWD with a diameter greater than 30 cm were in decay classes 3 and 4 (Table 1). We found 15 plant species growing on the CWD, and a further unidentified species (Suppl. material 6). Most species were infrequent and appeared in less than ten CWD, only Hemionitis farinosa (Forssk.) Christenh. (Pteridaceae) and Peperomia galioides Kunth (Piperaceae) appeared in more than nine CWD. Seven species were terrestrial plants, three were accidental epiphytes (terrestrial plants that accidentally grow as epiphytes), and four were true epiphytes. Most of the CWD (53.9%) had at least one plant growing on them, but only 25.8% had more than one species (Suppl. material 7). While most (75%) of the CWD in decay class 1 had no species growing on them, 74% of the CWD of decay class 4 had at least one plant species growing on them.

In the CWD, we found 83 Malaxis individuals and one Sarcoglottis schaffneri (Rchb.f.). Ames. Because most individuals were immature, had developing inflorescences, or were bearing fruit during sampling, we were unable to identify the Malaxis species. Orchids appeared on CWD from decay class 2 (14 Malaxis and one S. schaffneri individuals) to 4 (63 Malaxis individuals).

Among the 16 candidate models, the model including all variables (CWD diameter, CWD decay stage, species richness, and fungus coverage) was the best at explaining Malaxis abundance in CWD (Table 2). This model had a 73.0% probability of being the best (QAICc weight), and the closest model is separated from it by a QAIC distance greater than 2 (ΔQAICc). In the best model, the importance of the variables to predict the abundance of Malaxis follows the descending order: Species richness (coefficient = 1.58, p < 0.0001), diameter of the CWD (coefficient = 0.94, p < 0.00001), fungus coverage (coefficient = 0.68, p < 0.0001), and decay class (coefficient = 0.64, p < 0.0001). The coefficients indicate that Malaxis abundance increases as CWDs contains more species, are larger, and are in an advanced state of decay, with high fungal coverage.

Seed germination occurred after six weeks. By the seventh week, germination was observed in 70% of the pots with Malaxis brachyrrhynchos seeds and 75% of the pots with Stelis retusa seeds. The overall mean germination rate was 64% for M. brachyrrhynchos and 63% for S. retusa.

For M. brachyrrhynchos, seed germination was affected by each substrate type and by all interactions among substrates (Table 3). Except for the interaction between soil and CWD, the same occurred with Stelis retusa (Table 3). Among the main factors, S. retusa and M. brachyrrhynchos showed the same germination behaviour on bark and soil (Fig. 2A, B, E, F) (all the mean germination values are in Suppl. material 8). The germination was 2% (M. brachyrrhynchos) and 4% (S. retusa) higher in the presence of bark (Fig. 2A, B). The opposite occurs in the presence of soil, with germination diminished by 14% and 7% for M. brachyrrhynchos and S. retusa, respectively (Fig. 2E, F). For the CWD, the species showed different responses: in the presence of CWD, germination of M. brachyrrhynchos was 16% higher (Fig. 2C), whereas in S. retusa it was 2% lower (Fig. 2D).

Seed germination in the two-factor interactions (bark-CWD, soil-CWD, and soil-bark interactions) resembles the behaviour of the main factors (Fig. 3). In the interaction between bark and CWD (Fig. 3A, B), the germination rates of both species decreased when neither of these substrates were available, as well as when both substrates were present. The lower germination rates observed in the absence of bark and CWD can be attributed to reliance solely on soil as a substrate, which aligns with previous findings on the main factors. Conversely, the lower germination rates when both bark and CWD were present may indicate a preference for either substrate by each species. In the presence of the preferred substrate and in the absence of the other (Fig. 3A, B), the germination behaviour varied among the species. For S. retusa, germination was 6% lower in the presence of CWD and without bark (Fig. 3B), while for M. brachyrrhynchos, it was 12% higher (Fig. 3A).

The seed germination of S. retusa was similar in all four treatments involving soil-CWD interaction (Fig. 3F). On the other hand, CWD was found to have a positive effect on the germination of M. brachyrrhynchos seeds, as revealed by the highest germination rate when it was present (Fig. 3E). Conversely, the germination rate of these seeds decreased when the CWD was absent (Fig. 3E).

In the soil-bark interaction (Fig. 3C, D), the lowest germinations were found for both species when soil was present. Similarly to the single factors, seed germination of S. retusa increased with the bark and decreased when both substrates were absent (i.e. in the presence of CWD) (Fig. 3D); the opposite occurred in M. brachyrrhynchos (Fig. 3C).

Finally, seed germination in the interaction between the three substrates (Fig. 4) resembles the behaviour of the single main factors. Stelis retusa seed germination increased in the bark-only treatment (Fig. 4B), while germination of M. brachyrrhynchos increased in the CWD-only treatment (Fig. 4A).

In our study area, the accumulated basal area of dead standing trees (snags) is associated with Malaxis abundance and orchid richness, suggesting that forest patches with coarse, decaying wood favour the presence of orchids (Cruz-Fernández et al. 2011). However, the previous study did not measure the direct association between Malaxis and coarse wood debris. Our results show that CWD are scarce (< 90 CWD/ha, covering < 0.05% of the forest floor) and are biased toward small sizes. Few plant species could be identified growing in these CWD; some were fallen epiphytes and terrestrial plants that can grow in decayed wood. Among the terrestrial plants, we found Malaxis spp. and Sarcoglottis schaffneri orchids.

The best-predicting model shows that Malaxis abundance on CWD depends on plant species richness, CWD size, fungal coverage, and the CWD’s decaying state. Among these variables, fungus coverage is another indicator of CWD decay. Large CWD are infrequent; therefore, this critical resource is scarce. However, we recorded large Malaxis plants on the CWD. We did not observe dormant adults or protocorms, as has been done in other studies (Rasmussen and Whigham 1998), leaving open the possibility that the CWD are colonized early. Our data indicate the locations of adult Malaxis plants, and Malaxis establishment occurs when large CWD are in an early stage of decay.

We did not count the number of Malaxis plants on the forest floor because the substrate for germination of these individuals could not be inferred from their presence in the soil; they may germinate and establish on CWD, and when the CWD decomposes completely (i.e. becoming soil), the Malaxis plants remain as terrestrial plants. However, our germination experiment shows that decayed CWD is a key factor in the germination of Malaxis brachyrrhynchos.

The association between the abundance of Malaxis species on large CWD and CWD scarcity aligns with the first part of our hypothesis. We can conclude that one of the most critical factors limiting Malaxis abundance is the size of the CWD, and that decayed wood is a scarce substrate on the forest floor. Consequently, forest management practices that help maintain CWD on the forest floor will aid in orchid conservation (as opposed to CWD extraction for firewood or timber).

The general mean germination of Malaxis brachyrrhynchos and Stelis retusa was in the range reported in other orchid studies. For example, it was 89.1% in Corallorhiza odontorhiza (Willd.) Nutt., 77.8% in Dendrobium friedericksianum Rchb.f., 10.2% in Galearis spectabilis (L.) Raf., and 50.0% in Goodyera pubescens (Willd.) R.Br. (Rasmussen and Whigham 1993; Khamchatra et al. 2016). However, it was faster in M. brachyrrhynchos and S. retusa; we noted the first germination after six weeks, while in the previous species, the first germination was noted after 23 weeks (G. spectabilis), 24 weeks (G. pubescens), 30 weeks (C. odontorhiza), or even after seven months in D. friedericksianum and T. discolor (Rasmussen and Whigham 1993, 1998; Khamchatra et al. 2016). One possible reason is that in our ex situ experiment, we provided a more constant water supply to the seeds and used seeds coming from manual cross-pollination (Emeterio-Lara et al. 2018). In the in situ experiments previously mentioned, fruits were from natural pollination (including self-pollination), the water supply was not constant, and there was greater site variability (Brundrett et al. 2003).

Our ex situ experiment exposed random seed groups to the forest’s natural substrates, helping to discern the effects of these substrates on orchid germination (Swarts et al. 2010; McCormick and Jacquemyn 2014). These substrates accurately represent the same physical and microbiological conditions that seeds encounter in nature, including the community of orchid mycorrhizal fungi (Brundrett et al. 2003). The germination behaviours of Malaxis brachyrrhynchos and Stelis retusa correspond to their terrestrial and epiphytic habits, respectively. However, for both species, soil conditions adversely affect germination, suggesting that M. brachyrrhynchos on the forest floor depends on decayed coarse woody debris, and the epiphyte S. retusa on bark. It has been suggested that germination preferences between epiphytic and terrestrial orchids differ because of differences in seed size, seed water retention capacity, and the light and mycorrhizal fungi in the environment (Yoder et al. 2000, 2010; Swarts et al. 2010; Tsutsumi et al. 2011). It has been found that the composition of the mycorrhizal fungal community changes between different bark types (Ferrer and Gilbert 2003; Pecoraro et al. 2021), soil components (Brundrett et al. 2003), and between the arboreal and terrestrial environments. A new study should be conducted to identify the factors underlying the observed germination differences between S. retusa and M. brachyrrhynchos, especially regarding their associations with mycorrhizal fungi (Brundrett et al. 2003; Swarts et al. 2010; McCormick and Jacquemyn 2014; Khamchatra et al. 2016; Zhang et al. 2022; Rammitsu et al. 2023). However, for our hypothesis, it is clear that S. retusa seeds prefer to germinate on Quercus rugosa bark; meanwhile, M. brachyrrhynchos germinates preferentially on CWD, so its presence on the forest floor may be associated with the availability of decaying wood. It has been found that orchid mycorrhizal fungi differ across substrates, and the coarse organic matter in forest soil may have the greatest fungal activity, supporting orchid germination (Brundrett et al. 2003).

Our results support the idea that the regeneration niche may explain orchid rarity (Grubb 1977; Arditti and Ghani 2000; McCormick and Jacquemyn 2014; McCormick et al. 2018; Li et al. 2021). In this case, better germination of Malaxis brachyrrhynchos occurs on CWD and S. retusa in bark. Once the orchid seeds germinate, they do not develop a radicle, and several seedling growth phases may be recognized (Arditti 1967; Dressler 1981; Swarts et al. 2010; Khamchatra et al. 2016; Meng et al. 2019). Mortality occurs during the transitions between these phases (Li et al. 2021), reducing the final abundance. Our data clearly show that the substrate determines the number of seeds that can successfully germinate and proceed through subsequent growth phases.

Is the germination preference of M. brachyrrhynchos a phylogenetic signal from epiphytic ancestors? All the species in Malaxidinae descend from an epiphytic ancestor (Cameron 2005; Zhang et al. 2023). In our study area, M. brachyrrhynchos is a terrestrial species without reports of growing as an accidental or facultative epiphyte (Espejo-Serna et al. 2002; Cruz-Fernández et al. 2011; Mena-Jiménez et al. 2024). In other words, M. brachyrrhynchos could not be taken as a terrestrial plant evolving toward the canopy (a possible confounding interpretation in facultative epiphytes). Hence, we believe that the germination preference of M. brachyrrhynchos may be a phylogenetic signal. However, the substrate germination preferences of more Malaxidinae species must be determined before new research can be done to investigate the phylogenetic signal.

What does the germination behavior of terrestrial Malaxidinae orchids growing on coarse decayed wood suggest about orchid re-terrestrialization? Epiphytic orchid seeds may be selected for germinating on bark with the assistance of the mycorrhizal fungi associated with this substrate (Khamchatra et al. 2016; Rammitsu et al. 2023). However, the canopy also provides other substrates such as decaying wood (e.g. old bark, dead branches, branch crevices, snags), and re-terrestrialization may begin with a preference for these substrates. Once re-terrestrialized, these orchids must still require decayed wood for seed germination, as observed here and in Tipularia discolor (Rasmussen and Whigham 1993, 1998). In the terrestrial orchid T. discolor, seeds did not germinate in the soil unless woody debris was added (Rasmussen and Whigham 1993).

The high herbivory (Gaxiola et al. 2008) or mortality of fallen epiphytes on the forest floor (Matelson et al. 1993; Mondragón and Ticktin 2011; Orozco-Ibarrola et al. 2021; Spicer and Ortega 2023) suggests that natural selection may not favour re-terrestrialization. This high mortality may result from the actions of epiphyte enemies (i.e. the epiphyte enemy escape hypothesis; Gaxiola et al. 2008; Spicer and Ortega 2023), but a second explanation is that adult epiphytes are acclimatized to the crown microclimate and cannot automatically re-acclimatize to forest soil conditions. The lack of seed germination of T. discolor in forest soil (Rasmussen and Whigham 1993), and the low germination success of S. retusa and M. brachyrrhynchos in the soil-only treatment, suggest that the soil limits the germination success due to reasons like the lack of micorrhizal fungi or the presence of soil pathogens (i.e. supporting the enemy escape hypothesis). However, further research is required to confirm these. However, even when soil germination is lower, some seeds still germinate. This could be caused by woody debris in the soil of some pots, as previously suggested (Rasmussen and Whigham 1998), or it may indicate that orchids have more plasticity during germination (Li et al. 2021). In other studies, it has been found that the diversity of mycorrhizal fungi that facilitate the germination of orchid seeds is higher than that of those that assist plant development after germination (Khamchatra et al. 2016; Zhang et al. 2022; Rammitsu et al. 2023). Thus, the soil may lack mycorrhizal fungi that simultaneously assist the germination and development of some orchid species.