Skip to main content
  • ASM
    • Antimicrobial Agents and Chemotherapy
    • Applied and Environmental Microbiology
    • Clinical Microbiology Reviews
    • Clinical and Vaccine Immunology
    • EcoSal Plus
    • Eukaryotic Cell
    • Infection and Immunity
    • Journal of Bacteriology
    • Journal of Clinical Microbiology
    • Journal of Microbiology & Biology Education
    • Journal of Virology
    • mBio
    • Microbiology and Molecular Biology Reviews
    • Microbiology Resource Announcements
    • Microbiology Spectrum
    • Molecular and Cellular Biology
    • mSphere
    • mSystems
  • Log in
  • My alerts
  • My Cart

Main menu

  • Home
  • Articles
    • Current Issue
    • Accepted Manuscripts
    • COVID-19 Special Collection
    • Archive
    • Minireviews
  • For Authors
    • Submit a Manuscript
    • Scope
    • Editorial Policy
    • Submission, Review, & Publication Processes
    • Organization and Format
    • Errata, Author Corrections, Retractions
    • Illustrations and Tables
    • Nomenclature
    • Abbreviations and Conventions
    • Publication Fees
    • Ethics Resources and Policies
  • About the Journal
    • About AEM
    • Editor in Chief
    • Editorial Board
    • For Reviewers
    • For the Media
    • For Librarians
    • For Advertisers
    • Alerts
    • RSS
    • FAQ
  • Subscribe
    • Members
    • Institutions
  • ASM
    • Antimicrobial Agents and Chemotherapy
    • Applied and Environmental Microbiology
    • Clinical Microbiology Reviews
    • Clinical and Vaccine Immunology
    • EcoSal Plus
    • Eukaryotic Cell
    • Infection and Immunity
    • Journal of Bacteriology
    • Journal of Clinical Microbiology
    • Journal of Microbiology & Biology Education
    • Journal of Virology
    • mBio
    • Microbiology and Molecular Biology Reviews
    • Microbiology Resource Announcements
    • Microbiology Spectrum
    • Molecular and Cellular Biology
    • mSphere
    • mSystems

User menu

  • Log in
  • My alerts
  • My Cart

Search

  • Advanced search
Applied and Environmental Microbiology
publisher-logosite-logo

Advanced Search

  • Home
  • Articles
    • Current Issue
    • Accepted Manuscripts
    • COVID-19 Special Collection
    • Archive
    • Minireviews
  • For Authors
    • Submit a Manuscript
    • Scope
    • Editorial Policy
    • Submission, Review, & Publication Processes
    • Organization and Format
    • Errata, Author Corrections, Retractions
    • Illustrations and Tables
    • Nomenclature
    • Abbreviations and Conventions
    • Publication Fees
    • Ethics Resources and Policies
  • About the Journal
    • About AEM
    • Editor in Chief
    • Editorial Board
    • For Reviewers
    • For the Media
    • For Librarians
    • For Advertisers
    • Alerts
    • RSS
    • FAQ
  • Subscribe
    • Members
    • Institutions
Invertebrate Microbiology

Five-Year Cohort Study of Nosema spp. in Germany: Does Climate Shape Virulence and Assertiveness of Nosema ceranae?

Sebastian Gisder, Kati Hedtke, Nadine Möckel, Marie-Charlotte Frielitz, Andreas Linde, Elke Genersch
Sebastian Gisder
1Institute for Bee Research, Friedrich-Engels-Str. 32, D-16540 Hohen Neuendorf
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kati Hedtke
1Institute for Bee Research, Friedrich-Engels-Str. 32, D-16540 Hohen Neuendorf
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nadine Möckel
1Institute for Bee Research, Friedrich-Engels-Str. 32, D-16540 Hohen Neuendorf
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Marie-Charlotte Frielitz
1Institute for Bee Research, Friedrich-Engels-Str. 32, D-16540 Hohen Neuendorf
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Andreas Linde
2FH Eberswalde, Applied Ecology and Zoology, Alfred-Möller-Str. 1, D-16225 Eberswalde, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Elke Genersch
1Institute for Bee Research, Friedrich-Engels-Str. 32, D-16540 Hohen Neuendorf
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: elke.genersch@rz.hu-berlin.de
DOI: 10.1128/AEM.03097-09
  • Article
  • Figures & Data
  • Info & Metrics
  • PDF
Loading

ABSTRACT

Nosema ceranae and Nosema apis are two fungal pathogens belonging to the phylum Microsporidia and infecting the European honeybee, Apis mellifera. Recent studies have suggested that N. ceranae is more virulent than N. apis both at the individual insect level and at the colony level. Severe colony losses could be attributed to N. ceranae infections, and an unusual form of nosemosis is caused by this pathogen. In the present study, data from a 5-year cohort study of the prevalence of Nosema spp. in Germany, involving about 220 honeybee colonies and a total of 1,997 samples collected from these colonies each spring and autumn and analyzed via species-specific PCR-restriction fragment length polymorphism (RFLP), are described. Statistical analysis of the data revealed no relation between colony mortality and detectable levels of infection with N. ceranae or N. apis. In addition, N. apis is still more prevalent than N. ceranae in the cohort of the German bee population that was analyzed. A possible explanation for these findings could be the marked decrease in spore germination that was observed after even a short exposure to low temperatures (+4°C) for N. ceranae only. Reduced or inhibited N. ceranae spore germination at low temperatures should hamper the infectivity and spread of this pathogen in climatic regions characterized by a rather cold winter season.

Microsporidia are highly evolved fungi with an obligately intracellular parasitic lifestyle (14, 34). They are common parasites of insects and other invertebrates but are also known as parasites of vertebrates, including humans (8, 10, 47). To date, more than 160 genera and almost 1,300 species of microsporidia have been described in the literature (14), revealing a great diversity of morphology and life cycle strategies within this phylum. The common characteristics that qualify an organism as a microsporidian are that outside the host cell it exists only as metabolically inactive spores and that infection of a host cell involves spore germination, i.e., extrusion of a specialized structure, the polar tube, which pierces the host cell and inoculates infective sporoplasm directly into the cytoplasm of the host cell (9).

For honeybees, two species of microsporidia have been described: Nosema apis and Nosema ceranae (23, 49). Originally, it was assumed that N. apis was a pathogen specific for the European honeybee, Apis mellifera, causing nosemosis, while N. ceranae was specific for the Asian honeybee, Apis cerana. However, early cross-infection experiments demonstrated that N. apis can be infective for A. cerana and N. ceranae can successfully infect A. mellifera (21). The fact that N. ceranae can infect A. mellifera under natural conditions as well became evident when in 2005 N. ceranae was isolated from diseased honeybees (A. mellifera) in Taiwan (31) and was found in collapsing A. mellifera colonies in Spain (30). Many studies on the incidence of N. ceranae in A. mellifera were initiated due to these findings; they revealed a worldwide distribution of N. ceranae in A. mellifera populations (11, 12, 24, 30, 32, 33, 40, 42, 48). Experimental studies suggested that N. ceranae is highly virulent for A. mellifera (27), presumably due to immune suppression, which could be observed only after N. ceranae infection, not after N. apis infection (3). In the field, N. ceranae causes an unusual form of nosemosis, which led, and still leads, to severe colony losses in Spain (28, 37). One explanation for the higher virulence of N. ceranae in the field could be the better adaptation of N. ceranae than of N. apis to elevated temperatures (18, 36), indicating that N. ceranae might be a pathogen whose spread and assertiveness could be influenced by climate change.

Over the past decade, beekeepers in Europe and North America reported dramatic increases in colony losses, both during the season and over the winter (1, 2, 43, 46). The main culprits for these increases in colony mortality, among the pathogens affecting honeybee vitality, are viruses (6, 7, 15, 35) and Nosema ceranae (28, 37). While colony losses in the United States (termed colony collapse disorder [CCD] [45]) did not correlate with N. apis or N. ceranae infection (12, 15), colony mortality in Spain could clearly be attributed to N. ceranae infection (28, 37), and it was suggested that N. ceranae-induced colony collapse is not restricted to Spain but is at least a Europe-wide phenomenon. In Germany, beekeepers experienced dramatic overwintering losses in the winter of 2002 to 2003, and they have reported an increase in overwintering mortality since then. To evaluate whether or not N. ceranae can be correlated with these losses, we conducted a cohort study over 5 years involving 220 colonies in the northeastern part of Germany, and we determined colony mortality and the incidences and prevalences of the two Nosema species. We also analyzed individual bees for Nosema infection and Nosema spores for germination capacity. The implications of our results are discussed below.

MATERIALS AND METHODS

Bee samples and field survey.A cohort of 220 colonies kept in 22 apiaries (10 randomly selected colonies per apiary) and managed by hobbyist beekeepers in the northeastern part of Germany (see Fig. S1 in the supplemental material) were monitored for Nosema infection between spring 2005 and spring 2009. The colonies selected for the survey were closely monitored by a professional bee inspector twice a year for the duration of the study without introducing any changes in the beekeeping practices of the beekeepers. Each March, about 100 dead bees were collected from the bottom board of each colony (representing the bees that had died over the winter), and live bees were collected from the winter cluster (representing the bees that had survived the winter). Qualitative Nosema detection and differentiation did not differ between these two samples for a given colony; therefore, the bees collected from the bottom boards were used in the course of the study. At the end of September/beginning of October, around 100 live adult bees were collected from each colony (representing the bees that were raised for overwintering). The collected bees were frozen at −20°C and stored until analysis. Overwintering success, survival during the summer season, and the presence of symptoms of nosemosis were recorded individually for each colony. None of the monitored colonies showed clinical symptoms of nosemosis during the study. Lost colonies were replaced by colonies from the same apiary, preferentially by nuclei made from those colonies in the previous year. Since 220 colonies were observed throughout the study, the sampling of the monitored colonies and of nuclei replacing lost colonies at nine different time points between 2005 and 2009 resulted in a total of 1,997 samples analyzed (see Tables 1 and 2).

View this table:
  • View inline
  • View popup
TABLE 1.

Effects of Nosema infection in the spring on honeybee colony losses in the following season

View this table:
  • View inline
  • View popup
TABLE 2.

Effects of Nosema infection in the autumn on honeybee colony losses in the following winter

Detection of Nosema spp.Qualitative microscopic diagnosis of Nosema spores was performed according to the method described in the Manual of Diagnostic Tests and Vaccines for Terrestrial Animals, published by the Office International des Epizooties (OIE), the World Organization for Animal Health (38a). Briefly, 20 abdomens from each sample were homogenized together in 2 ml double-distilled H2O (ddH2O) and were checked by light microscopy (magnification, ×400) for the presence of microsporidian spores (41). The rather moderate sampling size does not allow the detection of the odd infected bee in the colony but does allow the detection of an infection level above ∼15% at the 5% significance level (22), which can be considered biologically relevant (28).

Microscopically positive homogenates were transferred to a 1.5-ml reaction tube and were thoroughly homogenized with a 3-mm-diameter tungsten carbide bead (Qiagen) in a mixer mill (Retsch) for 30 s at 30 Hz. Subsequently, the homogenate was centrifuged at 16,000 × g for 3 min, and DNA was extracted from the pellet using standard methods according to the manufacturer's protocols (Plant DNA extraction kit; Qiagen). The extracted DNA was resuspended in 50 μl elution buffer (Qiagen) and was frozen at −20°C until differentiation.

From each Nosema-positive bee sample, individual bees were dissected under a dissecting microscope (magnification, ×10) under sterile conditions. The tissues (gut, malpighian tubules, fat body, hypopharyngeal glands, and brain) were carefully isolated from individual bees by using fresh forceps for each organ to prevent contamination. The isolated organs were washed twice in 1× phosphate-buffered saline (PBS) and nuclease-free water to wash off any potentially contaminating hemolymph. Subsequently, DNA was extracted from the tissues as described above. The extracted DNA was resuspended in 50 μl elution buffer (Qiagen) and was frozen at −20°C until differentiation.

Differentiation of Nosema spp. via PCR-restriction fragment length polymorphism (PCR-RFLP).A region of the 16S rRNA gene that is conserved for N. apis and N. ceranae (33) was selected for primer design using MacVector, version 6.5 (Oxford Molecular). Primers nos-16S-fw (5′-CGTAGACGCTATTCCCTAAGATT-3′; positions 422 to 444 in GenBank accession no. U97150 [25a]) and nos-16S-rv (5′-CTCCCAACTATACAGTACACCTCATA-3′; positions 884 to 909 in GenBank accession no. U97150 [25a]) were used to amplify ca. 486 bp of the partial 16S rRNA gene.

PCR analysis was performed in a final volume of 25 μl containing 5 μl of template DNA (extracted from pelleted spores or individual organs), 2.5 μl of 10× Qiagen PCR buffer, 2.5 mM MgCl2, 200 μM each deoxynucleoside triphosphate (dNTP) (Qiagen), 0.625 U Taq polymerase (Qiagen), and 0.5 μM each forward and reverse primer. PCR parameters for amplification were as follows: an initial DNA denaturation of 5 min at 95°C; 45 cycles of 1 min at 95°C, 1 min at 53°C, and 1 min at 72°C; and a final extension step at 72°C for 4 min. Amplification products (5 μl DNA) were electrophoresed on 1.1% agarose gels (1× Tris-borate-EDTA [TBE]), stained with ethidium bromide, and visualized under UV light. A commercial 100-bp ladder (Peqlab) was used as a size marker. For each PCR, positive (reference N. apis and N. ceranae DNA extracts) and negative (ddH2O) controls were run along with DNA extracts of isolates as templates.

To differentiate between the species N. apis and N. ceranae, discriminating restriction endonuclease sites present in the PCR amplicon were used (33). The restriction endonuclease PacI provides one unique digestion site for N. ceranae, while the enzyme NdeI digests only N. apis. MspI digests both N. apis and N. ceranae and was used as a control for successful restriction digestion of PCR products.

The predicted restriction fragments produced from the digestion of the PCR amplicons are illustrated in Fig. S2 in the supplemental material. Amplicons were digested with MspI/PacI and with MspI/NdeI (New England Biolabs) in two reactions at 37°C for 3 h in order to analyze and confirm the presence of each Nosema species in each sample. Digestions were performed in a 12.5-μl volume with 7 μl of the amplified DNA and 1.5 U of each enzyme. For the reactions with MspI and NdeI, 1× NEBuffer 4 (provided by NEB with NdeI) was used as a buffer. For the reactions with MspI and PacI, 1× NEBuffer 1 plus bovine serum albumin (BSA) (provided by NEB with PacI) was used. Fragments were separated in a 3% NuSieve agarose gel (Cambrex Bio Science) in 1× TBE buffer with a 20-bp ladder as a size marker at 110 V for 1 h 30 min. Gels were stained with ethidium bromide and visualized under UV light.

Smear preparations for detection of developmental stages of Nosema spp.For the microscopic detection of developmental stages of Nosema spp. in smear preparations of bee tissue, adult bees infected with Nosema ceranae or Nosema apis were used. Brains, hypopharyngeal glands, malpighian tubules, fat bodies, and midguts were carefully isolated from individual bees as described above. Pieces of these tissues (area, 2 mm2) were placed on a microscopic slide for crush preparations using coverslips and the back of a pencil. After removal of the coverslips, the crushed tissue was air dried, fixed with 100% methanol, and air dried again prior to staining with Giemsa solution (1:10 Fluka Giemsa stain, modified solution) for 10 min. Stained tissues were rinsed with tap water, air dried, and embedded with Entellan (Merck). Microscopy analysis was performed at 1,000-fold magnification using a stereomicroscope (Leica).

In vitro germination of Nosema spores.Bee samples that tested positive for either N. apis or N. ceranae were used for isolation of the respective spores. The procedure for spore accumulation and purification was modified from the method of Chen et al. (13). Alimentary tracts of 20 individual bees were removed by snatching the sting with forceps and gently pulling the hindgut and the midgut out of the abdomen. The sting was cut with a scalpel, and the alimentary tract was transferred to a 1.5-ml reaction tube and crushed in a final volume of 1.5 ml sterile double-distilled water in a mixer mill (Retsch) with a 3-mm-diameter tungsten carbide bead (Qiagen) for 30 s at 30 Hz. The homogenate was filtered through a nylon cell strainer (Falcon) with a 100-μm mesh diameter. The filtrate was gently overlaid on a 90% Percoll (Sigma-Aldrich) cushion in a 15-ml reaction tube and was centrifuged twice at 15,000 × g for 45 min at 20°C in a Eppendorf 5810 R centrifuge using an F34-6-38 rotor. The small but dense band just above the bottom of the tube, which contained purified spores, was aspirated with a syringe and a Sterican 0.8-mm by 120-mm needle (Braun) and was transferred to a 15-ml reaction tube. Spores were washed three times with 8 volumes of double-distilled water and were centrifuged at 6,500 × g for 10 min at 20°C. After a final centrifugation step, the supernatant was removed, the pellet was resuspended in 500 μl AE buffer (Qiagen), and the spore concentration was determined using a hemocytometer. The identity of the spores was verified by PCR-RFLP as described above. Aliquots of purified N. apis or N. ceranae spores (20 μl; 2E+8 spores per ml) were spotted onto glass slides, air dried, and kept at different temperatures for different periods of time. Subsequently, germination was triggered by the addition of 30 μl of 0.1 M sucrose in PBS buffer to the air-dried spores (39), a procedure that mimics the natural conditions for the germination of environmental spores. Representative results obtained with freshly isolated, dried, and germinated spores (+22°C) or after storage of the dried spores at 4°C for 4 days are shown in Fig. 4. These experiments were repeated 10 times with different spore preparations (n = 10).

Data evaluation and statistical analysis.We compared surviving and collapsed colonies by using nonparametric chi-square tests, because the basal assumptions of parametric tests (i.e., normality and constant variance) were not satisfied. The distribution of colony losses differed between the years and between the seasons (see Tables 1 and 2); therefore, the data sets were analyzed separately. Chi-square tests were performed by comparing the infection statuses of the surviving colonies with those of the collapsed colonies. A P value of <0.05 was considered significant.

RESULTS

Incidences of Nosema apis and Nosema ceranae.During the entire period of our study, we observed the seasonality of Nosema-positive colonies that was described previously (20) (Fig. 1). The proportion of samples positive for Nosema spp. in the cohort monitored was always higher in the spring than in the autumn. In the spring, the proportion of Nosema-positive samples ranged from 22.4% (2007) to 35.4% (2008). In the autumn, the highest prevalence of Nosema spp. was detected in 2005 (12.7%) and the lowest in 2008 (5.2%).

FIG. 1.
  • Open in new tab
  • Download powerpoint
FIG. 1.

Epidemiological survey of 22 apiaries in the northeastern part of Germany for Nosema prevalence between spring 2005 and spring 2009. The prevalences of samples positive for Nosema spp. (filled diamonds), Nosema apis (open squares), Nosema ceranae (open triangles), and both species (mixed infections) (shaded circles) are shown at each time point. Diagnosis of Nosema spp. was performed microscopically. Nosema differentiation was performed via RFLP analysis of a PCR-amplified partial sequence of the 16S rRNA gene.

Since recent studies had suggested the replacement of N. apis by N. ceranae in Europe (33, 40), we expected to find hardly any N. apis-positive samples. Surprisingly, this was not the case. With the exception of spring 2007, N. apis was always more prevalent than N. ceranae. The proportion of N. apis-positive colonies in the spring ranged from 15.7% (2009) to 3.7% (2007), and that in the autumn ranged from 8.0% (2005) to 2.9% (2008) (Fig. 1). The proportion of colonies that tested positive for N. ceranae ranged from 14.9% (2007) to 4.1% (2005) in the spring and from 4.2% (2005) to 1.3% (2006) in the autumn (Fig. 1). In spring 2007, 14.9% N. ceranae-positive colonies and only 3.1% N. apis-positive colonies were detected in our cohort. A low prevalence of mixed infections with the same seasonal pattern (more positive colonies in the spring than in the autumn) was also detected throughout the study period (Fig. 1).

Colony losses during the study period.In order to evaluate the impact of Nosema infection on the mortality of honeybee colonies, we differentiated between colony losses in the summer season (weeks 15 to 35) and overwintering losses (weeks 36 to week 14 of the following year), and we related these to the detection of Nosema spp., N. apis, and N. ceranae in spring and autumn samples (Tables 1 and 2). The winter losses we observed in our cohort during the study period ranged from 22.4% (2005/2006) to 4.8% (2008/2009), showing that our study period covered winters with low to normal colony mortality rates as well as winters with rather high colony mortality rates (Table 2). We observed a similar variation, although on a much lower level (1.8% for 2007; 6.7% for 2008), for colony losses during the season (Table 1). Our data did not reveal a significant relation between the detection of Nosema infection in the spring and colony losses in the following season (Table 1, P values) or between detectable Nosema infections in the autumn and colony collapses during the following winter (Table 2, P values). In addition, no significant differences between the mortality rates of uninfected colonies and those of colonies infected by N. apis, N. ceranae, or both (mixed infections) could be established (P, ≫0.05).

Tissue tropism of N. apis and N. ceranae.While N. apis infections are reported to be restricted to the gut (16, 19, 21), N. ceranae could also be detected in hypopharyngeal glands, salivary glands, malpighian tubules, and fat bodies, indicating a more generalized infection (13), which could possibly be linked to the reported increased lethality of N. ceranae (27, 40). Since our results did not reveal the previously reported, and therefore expected, correlation between N. ceranae infection and colony losses (28, 37), we asked whether a less virulent form of N. ceranae, which did not cause such generalized infections, might be present in Germany. We therefore analyzed the tissue tropism of N. apis and N. ceranae in the infected bees collected in our study in detail but did not find any substantial differences from what had already been described (13). By the use of PCR analysis and RFLP differentiation, N. apis was detected only in the guts of infected bees, supporting the tissue tropism reported for N. apis (16, 17, 19). In contrast, using the same method, we detected N. ceranae in hypopharyngeal glands, brains, guts, malpighian tubules, and fat bodies (Fig. 2). For the majority of the bees, infection of the gut could be demonstrated. In addition to the detection of N. ceranae in the tissues, as reported by Chen and coworkers (13), we could detect N. ceranae in the brain as well. All N. ceranae-positive bees analyzed showed a rather generalized infection, with at least two different tissues testing positive for N. ceranae. However, using the less sensitive method of Giemsa-stained smear preparations, we could detect both N. apis and N. ceranae only in gut tissue (Fig. 3).

FIG. 2.
  • Open in new tab
  • Download powerpoint
FIG. 2.

Detection of Nosema spp. and differentiation via species-specific 16S rRNA gene RFLP in different tissues of infected bees sampled from colonies that tested positive for Nosema ceranae only. The tissues analyzed were hypopharyngeal glands (h. glands), brains, guts, malpighian tubules (m. tubules), and fat bodies.

FIG. 3.
  • Open in new tab
  • Download powerpoint
FIG. 3.

Detection of Nosema spp. via smear preparations in infected bees. Different organs of infected bees were analyzed for the presence of spores and vegetative stages of Nosema spp. by using Giemsa-stained smear preparations. A representative image from the gut of an N. apis-infected bee is shown. n, nucleus. Nuclei are stained red, and spores are stained blue-white.

Germination of N. apis and N. ceranae.Previous studies have shown that N. ceranae spores, but not N. apis spores, are sensitive to a temperature of −20°C (18). Since germination is the first step in the infection process, we analyzed the germination capacities of N. apis and N. ceranae spores exposed to low temperatures versus room temperature. We found that the germination of both Nosema species was affected after 4 days at 4°C. Estimation of the numbers of germinated spores and extruded polar tubes revealed that about 80% of the N. apis spores were still capable of germination, while N. ceranae spore germination was reduced to less than 10% (Fig. 4). In addition, the polar tubes extruded from the few N. ceranae spores still capable of germination were unusually short, most likely not representing true germination (Fig. 4).

FIG. 4.
  • Open in new tab
  • Download powerpoint
FIG. 4.

In vitro germination of N. apis and N. ceranae spores. Spores were isolated, air dried on glass slides, and kept at different temperatures for different periods. Representative results obtained after storage of the dried spores at +4°C for 4 days are shown in comparison with freshly isolated, dried, and germinated spores. Extruded polar tubes can be seen as curved lines in the images. White arrowheads point to extremely short, extruded polar tubes seen only for N. ceranae after 4 days at +4°C.

DISCUSSION

A recent study (33) is frequently cited to substantiate the notion that N. ceranae is predominant in Europe, although the published data rather show that the patterns of incidence of N. apis and N. ceranae differ for different regions in Europe. N. ceranae seems to be more prevalent than N. apis in Denmark, Greece, Italy, Serbia, and Spain, while N. apis could be detected more often than N. ceranae in Sweden and the United Kingdom. In the German samples analyzed (14 random samples from spring 2006), the prevalence of both Nosema species was balanced. Although in that study a meaningful overview was given for Europe, the limited data for each country lower the representative character of the study. The present cohort study, performed with about 220 colonies over 5 years, reveals that N. apis is still more prevalent in Germany, as in the United Kingdom and Sweden (33), than N. ceranae, although the prevalence of N. ceranae might be on the rise. Two- and 3-fold increases in the proportion of N. ceranae positive colonies could be seen in spring 2008 and spring 2009, respectively, over that at the beginning of the study (spring 2005 and spring 2006), suggesting a current process of increasing N. ceranae prevalence. We will continue our study with the same cohort in order to see whether or not we are observing the replacement of N. apis by N. ceranae in Germany, a process that seems likely according to data from other European countries (33) and the United States (12) and that will be most interesting to watch.

One of the hallmarks of the unusual form of nosemosis caused by N. ceranae is reported to be a “loss of seasonality.” For N. apis it has been reported that in spring more colonies will have detectable infection levels, i.e., more individuals will be infected, and infected individuals will exhibit a higher spore load. Typically, pathological symptoms of nosemosis (dysentery accompanied by defecation within the hive; crawling bees) will be evident in early spring, and colonies will collapse before the season starts (4, 5, 20). N. ceranae-infected European honeybees do not show this pathology. Instead, it is reported that colonies eventually collapse from the disease without showing any obvious symptoms of nosemosis (24). In addition, for N. ceranae infections of European honeybees in Spain, a change in seasonality has been reported, with an increase in Nosema-positive samples throughout the year, finally leading to a total absence of seasonality in infection prevalence (37). Our data do not support such a situation for the prevalence of Nosema-positive colonies in Germany. The proportion of colonies with detectable levels of Nosema infection was always higher in spring than in autumn. For colonies infected by N. ceranae, seasonality was not really pronounced in the beginning of the study, when the proportion of infected colonies was quite low (about 4%), but this changed in 2007, when 14.9% of the spring samples tested positive for N. ceranae. Since then, N. ceranae has followed the seasonality that is described for N. apis and could be verified in our study except for one season, spring 2007, when as few as 3.1% of the colonies were N. apis positive. Hence, the notion that the prevalence of colonies with detectable levels of N. ceranae infection over the year does not show the “normal” seasonality could not be substantiated in our study.

Studies performed with experimental infection of caged bees have recently suggested that in addition to a different pathology, N. ceranae has a higher individual virulence than N. apis (27, 40), although this effect could be overcome by feeding the caged bees ad libitum (38). Reports on the colony-level virulence of N. ceranae are contradictory. Several studies from Spain suggest that N. ceranae is highly virulent at the colony level and that hence, infected colonies inevitably die from the infection if left untreated. These studies also imply that N. ceranae is the cause for the unusual colony losses reported from several regions in Europe and in the United States (28, 29, 37). Other studies rather question a link between N. ceranae infections and increased colony mortality or identify other causes for unusual colony losses (11, 12, 15, 32, 44, 45). The results of our study also failed to reveal a relation between N. ceranae infection of colonies and colony mortality, even in seasons with unusually high colony loss rates. Likewise, monitoring of the fate of individual N. ceranae-infected colonies over several years (data not shown) did not show a mandatory link between this infection and failure of the colony. Instead, infestation by Varroa destructor and infection with deformed wing virus (DWV) and acute bee paralysis virus (ABPV) could be identified as pathogenic processes related with high significance to increased colony mortality in Germany (26).

Nevertheless, the facts remain that (i) in Spain and Italy N. ceranae nearly replaced N. apis over the past decade (33) and (ii) N. ceranae infections cause severe honeybee colony losses, at least in Spain (28, 30, 37). Possible reasons for this assertiveness and virulence of N. ceranae are the exceptional biotic potential of N. ceranae at higher temperatures and the spores' tolerance to temperatures as high as 60°C, combined with resistance to desiccation (18, 36). On the other hand, N. ceranae spores have been reported to be sensitive to freezing temperatures (18, 20). Since in our study, unexpectedly, neither predominance nor a noticeable colony-level virulence of N. ceranae could be verified, we searched for an explanation. Differences in virulence between genetically different isolates (25) were possible and will be addressed in further studies. However, in view of the studies of the influence of temperature on N. ceranae spore viability and infectivity, discussed above, we rather suspected that climatic factors play a role in the outcome of N. ceranae infections (20). The northeastern part of Germany is characterized by moderately warm but not hot summers and rather cold and long winters (December to February) with a mean upper temperature threshold of about +4°C. We therefore analyzed and compared the germination capacities of N. apis and N. ceranae spores at different temperatures. After as few as 4 days at +4°C, we found a marked decrease in the level of germination of N. ceranae spores but not in that of N. apis spores. Since germination is the first step in the infection process of ingested environmental spores, a decrease in the germination of N. ceranae spores after such a short time at moderately cold temperatures will reduce the infectivity and virulence of N. ceranae at low temperatures. This cold sensitivity of N. ceranae spores, not seen with N. apis spores, could pose a clear disadvantage for N. ceranae in competing with N. apis in climatic regions with comparable or even colder temperatures in the winter season. Environmental spores (e.g., those defecated inside the hive and “waiting” to be ingested by another bee) of N. ceranae will have a limited chance of staying infective during any cold period in winter, while N. apis spores might stay unaffected even at −20°C (20). Within-colony transmission of N. ceranae will, therefore, be hampered at lower temperatures, preventing N. ceranae from building up in the colony over the winter as N. apis does. The disadvantage posed by the cold sensitivity of N. ceranae spores might even neutralize any advantageous effect of thermotolerance and resistance to desiccation, especially in regions where the summer temperature rarely rises above 33°C. This explanation for the limited assertiveness and virulence of N. ceranae in Germany so far is consistent with earlier results showing a predominance of N. apis in countries such as Sweden (with rather cold and long winters) but a prevalence of N. ceranae in countries such as Italy and Spain (with hot summers and moderate winters) (33). Further laboratory and field studies are needed to substantiate the impact of climate on the colony-level virulence of the two Nosema species and on the success of N. ceranae in replacing N. apis in the European honeybee population.

In summary, N. ceranae is still not predominant in Germany and has not yet replaced N. apis in the European honeybee population in Germany. No link between N. ceranae infection and an increased risk of colony mortality could be established. We presented evidence that the germination capacity of N. ceranae spores is affected by moderately low temperatures (+4°C), which may hamper the infectivity and transmission of N. ceranae during the winter season. Hence, we suggest that the virulence and assertiveness of N. ceranae in the European honeybee population is influenced by climate and might change with climatic changes.

ACKNOWLEDGMENTS

This work was supported by grants from the Ministries for Agriculture from the Federal States of Brandenburg (MIL) and Sachsen-Anhalt (MLU), Germany, and through grants from Bayer Animal Health AG. S.G. and N.M. were supported by a grant from the German Ministry of Nutrition, Agriculture and Consumer Protection (BMELV).

FOOTNOTES

    • Received 22 December 2009.
    • Accepted 1 March 2010.
  • Copyright © 2010 American Society for Microbiology

REFERENCES

  1. 1.↵
    Aizen, M. A., L. A. Garibaldi, S. A. Cunningham, and A. M. Klein. 2008. Long-term global trends in crop yield and production reveal no current pollination shortage but increasing pollinator dependency. Curr. Biol.18:1572-1575.
    OpenUrlCrossRefPubMedWeb of Science
  2. 2.↵
    Aizen, M. A., and L. D. Harder. 2009. The global stock of domesticated honey bees is growing slower than agricultural demand for pollination. Curr. Biol.19:915-918.
    OpenUrlCrossRefPubMedWeb of Science
  3. 3.↵
    Antúnez, K., R. Martin-Hernandez, L. Prieto, A. Meana, P. Zunino, and M. Higes. 2009. Immune suppression in the honey bee (Apis mellifera) following infection by Nosema ceranae (Microsporidia). Environ. Microbiol.11:2284-2290.
    OpenUrlCrossRefPubMedWeb of Science
  4. 4.↵
    Bailey, L. 1955. The epidemiology and control of Nosema disease of the honey bee. Ann. Appl. Biol.43:379-389.
    OpenUrlCrossRef
  5. 5.↵
    Bailey, L. 1967. Nosema apis and dysentery of the honey bee. J. Apicult. Res.6:121-125.
    OpenUrl
  6. 6.↵
    Ball, B. V. 1983. The association of Varroa jacobsoni with virus diseases of honey bees. Exp. Appl. Acarol.19:607-613.
    OpenUrlCrossRef
  7. 7.↵
    Ball, B. V. 1996. Honey bee viruses: a cause for concern? Bee World77:117-119.
    OpenUrl
  8. 8.↵
    Becnel, J. J., and T. G. Andreadis. 1999. Microsporidia in insects, p. 447-501. In M. Wittner and L. M. Weiss (ed.), The microsporidia and microsporidiosis. ASM Press, Washington, DC.
  9. 9.↵
    Bigliardi, E., and L. Sacchi. 2001. Cell biology and invasion of the microsporidia. Microbes Infect.3:373-379.
    OpenUrlCrossRefPubMedWeb of Science
  10. 10.↵
    Canning, E. U., and J. Lom. 1986. The microsporidia of vertebrates, p. 1-16. Academic Press, New York, NY.
  11. 11.↵
    Chauzat, M. P., M. Higes, R. Martín-Hernandez, A. Meana, N. Cougoule, and J. P. Faucon. 2007. Presence of Nosema ceranae in French honey bee colonies. J. Apicult. Res.46:127-128.
    OpenUrl
  12. 12.↵
    Chen, Y., J. D. Evans, I. B. Smith, and J. S. Pettis. 2008. Nosema ceranae is a long-present and wide-spread microsporidian infection of the European honey bee (Apis mellifera) in the United States. J. Invertebr. Pathol.97:186-188.
    OpenUrlCrossRefPubMed
  13. 13.↵
    Chen, Y. P., J. D. Evans, C. Murphy, R. Gutell, M. Zuker, D. Gundensen-Rindal, and J. S. Pettis. 2009. Morphological, molecular, and phylogenetic characterization of Nosema ceranae, a microsporidian parasite isolated from the European honey bee, Apis mellifera. J. Eukaryot. Microbiol.56:142-147.
    OpenUrlCrossRefPubMed
  14. 14.↵
    Corradi, N., and P. J. Keeling. 2009. Microsporidia: a journey through radical taxonomical revisions. Fungal Biol. Rev.23:1-8.
    OpenUrlCrossRef
  15. 15.↵
    Cox-Foster, D. L., S. Conlan, E. C. Holmes, G. Palacios, J. D. Evans, N. A. Moran, P.-L. Quan, S. Briese, M. Hornig, D. M. Geiser, V. Martinson, D. vanEngelsdorp, A. L. Kalkseitn, L. Drysdale, J. Hui, J. Zhai, L. Cui, S. Hutchison, J. F. Simons, M. Egholm, J. S. Pettis, and W. I. Lipkin. 2007. A metagenomic survey of microbes in honey bee colony collapse disorder. Science318:283-287.
    OpenUrlAbstract/FREE Full Text
  16. 16.↵
    de Graaf, D., and F. J. Jacobs. 1991. Tissue specificity of Nosema apis. J. Invertebr. Pathol.58:277-278.
    OpenUrlCrossRef
  17. 17.↵
    de Graaf, D., H. Raes, G. Sabbe, P. H. Rycke, and F. J. Jacobs. 1994. Early development of Nosema apis (Microspora: Nosematidae) in the midgut epithelium of the honeybee (Apis mellifera). J. Invertebr. Pathol.63:74-81.
    OpenUrlCrossRef
  18. 18.↵
    Fenoy, S., C. Rueda, M. Higes, R. Martín-Hernandez, and C. del Aguila. 2009. High-level resistance of Nosema ceranae, a parasite of the honeybee, to temperature and desiccation. Appl. Environ. Microbiol.75:6886-6889.
    OpenUrlAbstract/FREE Full Text
  19. 19.↵
    Fries, I. 1988. Infectivity and multiplication of Nosema apis Z. in the ventriculus of the honey bee. Apidologie19:319-328.
    OpenUrlCrossRef
  20. 20.↵
    Fries, I. 2010. Nosema ceranae in European honey bees (Apis mellifera). J. Invertebr. Pathol.103:573-579.
    OpenUrl
  21. 21.↵
    Fries, I. 1997. Protozoa, p. 59-76. In R. A. Morse and K. Flottum (ed.), Honey bee pests, predators, and diseases. A. I. Root Company, Medina, OH.
  22. 22.↵
    Fries, I., G. Ekbohm, and E. Villumstad. 1984. Nosema apis, sampling techniques and honey yield. J. Apicult. Res.23:102-105.
    OpenUrl
  23. 23.↵
    Fries, I., F. Feng, A. daSilva, S. B. Slemenda, and N. J. Pieniazek. 1996. Nosema ceranae n. sp. (Microspora, Nosematidae), morphological and molecular characterization of a microsporidian parasite of the Asian honey bee Apis cerana (Hymenoptera, Apidae). Eur. J. Protistol.32:356-365.
    OpenUrlCrossRef
  24. 24.↵
    Fries, I., R. Martin, A. Meana, P. Garcia-Palencia, and M. Higes. 2006. Natural infections of Nosema ceranae in European honey bees. J. Apicult. Res.45:230-233.
    OpenUrl
  25. 25.↵
    Gatehouse, H. S., and L. A. Malone. 1999. Genetic variability among Nosema apis isolates. J. Apicult. Res.38:79-85.
    OpenUrl
  26. 25a.↵
    Gatehouse, H. S., and L. A. Malone. 1998. The ribosomal RNA gene region of Nosema apis (Microspora): DNA sequence for small and large subunit rRNA genes and evidence of a large tandem repeat unit size. J. Invertebr. Pathol.71:97-105.
    OpenUrlCrossRefPubMedWeb of Science
  27. 26.↵
    Genersch, E., W. von der Ohe, H. Kaatz, A. Schroeder, C. Otten, R. Büchler, S. Berg, W. Ritter, W. Mühlen, S. Gisder, M. Meixner, G. Liebig, and P. Rosenkranz. The German bee monitoring project: a long term study to understand periodically high winter losses of honey bee colonies. Apidologie, in press. doi:10.1051/apido/2010014.
    OpenUrlCrossRef
  28. 27.↵
    Higes, M., P. Garcia-Palencia, R. Martín-Hernandez, and A. Meana. 2007. Experimental infection of Apis mellifera honeybees with Nosema ceranae (Microsporidia). J. Invertebr. Pathol.94:211-217.
    OpenUrlCrossRefPubMed
  29. 28.↵
    Higes, M., R. Martín-Hernández, C. Botías, E. Garrido Bailón, A. V. González-Porto, L. Barrios, M. J. del Nozal, J. L. Bernal, J. J. Jiménez, P. García Palencia, and A. Meana. 2008. How natural infection by Nosema ceranae causes honeybee colony collapse. Environ. Microbiol.10:2659-2669.
    OpenUrlCrossRefPubMedWeb of Science
  30. 29.↵
    Higes, M., R. Martín-Hernandez, C. Botias, and A. Meana. 2009. The presence of Nosema ceranae (Microsporidia) in African honey bees (Apis mellifera intermissa). J. Apicult. Res.48:217-219.
    OpenUrlCrossRef
  31. 30.↵
    Higes, M., R. Martin, and A. Meana. 2006. Nosema ceranae, a new microsporidian parasite in honeybees in Europe. J. Invertebr. Pathol.92:93-95.
    OpenUrlCrossRefPubMedWeb of Science
  32. 31.↵
    Huang, W. F., J. H. Jiang, Y. W. Chen, and C. H. Wang. 2007. A Nosema ceranae isolate from the honeybee Apis mellifera. Apidologie38:30-37.
    OpenUrlCrossRef
  33. 32.↵
    Invernizzi, C., C. Abud, I. H. Tomasco, J. Harriet, G. Ramallo, J. Campá, H. Katz, G. Gardiol, and Y. Mendoza. 2009. Presence of Nosema ceranae in honeybees (Apis mellifera) in Uruguay. J. Invertebr. Pathol.101:150-153.
    OpenUrlCrossRefPubMed
  34. 33.↵
    Klee, J., A. M. Besana, E. Genersch, S. Gisder, A. Nanetti, D. Q. Tam, T. X. Chinh, F. Puerta, J. M. Ruz, P. Kryger, D. Message, F. Hatjina, S. Korpela, I. Fries, and R. J. Paxton. 2007. Widespread dispersal of the microsporidian Nosema ceranae, an emergent pathogen of the western honey bee, Apis mellifera. J. Invertebr. Pathol.96:1-10.
    OpenUrlCrossRefPubMed
  35. 34.↵
    Lee, S. C., N. Corradi, E. J. Byrnes III, S. Torres-Martinez, F. S. Dietrich, P. J. Keeling, and J. Heitman. 2008. Microsporidia evolved from ancestral sexual fungi. Curr. Biol.18:1675-1679.
    OpenUrlCrossRefPubMedWeb of Science
  36. 35.↵
    Martin, S. J. 2001. The role of Varroa and viral pathogens in the collapse of honeybee colonies: a modelling approach. J. Appl. Ecol.38:1082-1093.
    OpenUrlCrossRefWeb of Science
  37. 36.↵
    Martín-Hernández, R., A. Meana, P. Garcia-Palencia, P. Marin, C. Botias, E. Garrido-Bailon, L. Barrios, and M. Higes. 2009. Effect of temperature on the biotic potential of honeybee microsporidia. Appl. Environ. Microbiol.75:2554-2557.
    OpenUrlAbstract/FREE Full Text
  38. 37.↵
    Martín-Hernández, R., A. Meana, L. Prieto, A. M. Salvador, E. Garrido-Bailon, and M. Higes. 2007. Outcome of colonization of Apis mellifera by Nosema ceranae. Appl. Environ. Microbiol.73:6331-6338.
    OpenUrlAbstract/FREE Full Text
  39. 38.↵
    Mayack, C., and D. Naug. 2009. Energetic stress in the honeybee Apis mellifera from Nosema ceranae infection. J. Invertebr. Pathol.100:185-188.
    OpenUrlCrossRefPubMedWeb of Science
  40. 38a.↵
    Office Internationale des Epizooties—World Organization for Animal Health. 2008. Manual of diagnostic tests and vaccines for terrestrial animals, p. 410-414. OIE, Paris, France.
  41. 39.↵
    Olsen, P. E., W. A. Rice, and T. P. Liu. 1986. In vitro germination of Nosema apis spores under conditions favorable for the generation and maintenance of sporoplasms. J. Invertebr. Pathol.47:65-73.
    OpenUrlCrossRef
  42. 40.↵
    Paxton, R. J., J. Klee, S. Korpela, and I. Fries. 2007. Nosema ceranae has infected Apis mellifera in Europe since at least 1998 and may be more virulent than Nosema apis. Apidologie38:558-565.
    OpenUrlCrossRef
  43. 41.↵
    Ritter, W. 1996. Diagnostik und Bekämpfung der Bienenkrankheiten. Fischer, Stuttgart, Germany.
  44. 42.↵
    Tapaszti, Z., P. Forgách, C. Kovágó, L. Békési, T. Bakonyi, and M. Rusvai. 2009. First detection and dominance of Nosema ceranae in Hungarian honeybee colonies. Acta Vet. Hung.57:383-388.
    OpenUrlCrossRefPubMed
  45. 43.↵
    vanEngelsdorp, D. 2008. A survey of honey bee colony losses in the U.S., fall 2007 to spring 2008. PLoS One3:e4071.
    OpenUrlCrossRefPubMed
  46. 44.↵
    vanEngelsdorp, D., J. D. Evans, L. Donovall, C. Mullin, M. Frazier, J. Frazier, D. R. Tarpy, J. Hayes, and J. S. Pettis. 2009. “Entombed pollen”: a new condition in honey bee colonies associated with increased risk of colony mortality. J. Invertebr. Pathol.101:147-149.
    OpenUrlCrossRefPubMed
  47. 45.↵
    vanEngelsdorp, D., J. D. Evans, C. Saegerman, C. Mullin, E. Haubruge, B. K. Nguyen, M. Frazier, J. Frazier, D. Cox-Foster, Y. Chen, R. Underwood, D. R. Tarpy, and J. S. Pettis. 2009. Colony collapse disorder: a descriptive study. PLoS One4:e6481.
    OpenUrlCrossRefPubMed
  48. 46.↵
    vanEngelsdorp, D., and M. D. Meixner. 2010. A historical review of managed honey bee populations in Europe and the United States and the factors that may affect them. J. Invertebr. Pathol.103(Suppl. 1):S80-S95.
    OpenUrlCrossRefPubMed
  49. 47.↵
    Weber, R., R. T. Bryan, D. A. Schwartz, and R. L. Owen. 1994. Human microsporidial infections. Clin. Microbiol. Rev.7:426-461.
    OpenUrlAbstract/FREE Full Text
  50. 48.↵
    Williams, G. R., A. B. A. Shafer, R. E. L. Rogers, D. Shutler, and D. T. Stewart. 2008. First detection of Nosema ceranae, a microsporidian parasite of European honey bees (Apis mellifera), in Canada and central USA. J. Invertebr. Pathol.97:189-192.
    OpenUrlCrossRefPubMed
  51. 49.↵
    Zander, E. 1909. Tierische Parasiten als Krankheitserreger bei der Biene. Münch. Bienenzeitung31:196-204.
    OpenUrl
PreviousNext
Back to top
Download PDF
Citation Tools
Five-Year Cohort Study of Nosema spp. in Germany: Does Climate Shape Virulence and Assertiveness of Nosema ceranae?
Sebastian Gisder, Kati Hedtke, Nadine Möckel, Marie-Charlotte Frielitz, Andreas Linde, Elke Genersch
Applied and Environmental Microbiology Apr 2010, 76 (9) 3032-3038; DOI: 10.1128/AEM.03097-09

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Print

Alerts
Sign In to Email Alerts with your Email Address
Email

Thank you for sharing this Applied and Environmental Microbiology article.

NOTE: We request your email address only to inform the recipient that it was you who recommended this article, and that it is not junk mail. We do not retain these email addresses.

Enter multiple addresses on separate lines or separate them with commas.
Five-Year Cohort Study of Nosema spp. in Germany: Does Climate Shape Virulence and Assertiveness of Nosema ceranae?
(Your Name) has forwarded a page to you from Applied and Environmental Microbiology
(Your Name) thought you would be interested in this article in Applied and Environmental Microbiology.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Five-Year Cohort Study of Nosema spp. in Germany: Does Climate Shape Virulence and Assertiveness of Nosema ceranae?
Sebastian Gisder, Kati Hedtke, Nadine Möckel, Marie-Charlotte Frielitz, Andreas Linde, Elke Genersch
Applied and Environmental Microbiology Apr 2010, 76 (9) 3032-3038; DOI: 10.1128/AEM.03097-09
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Top
  • Article
    • ABSTRACT
    • MATERIALS AND METHODS
    • RESULTS
    • DISCUSSION
    • ACKNOWLEDGMENTS
    • FOOTNOTES
    • REFERENCES
  • Figures & Data
  • Info & Metrics
  • PDF

KEYWORDS

Bees
Nosema

Related Articles

Cited By...

About

  • About AEM
  • Editor in Chief
  • Editorial Board
  • Policies
  • For Reviewers
  • For the Media
  • For Librarians
  • For Advertisers
  • Alerts
  • RSS
  • FAQ
  • Permissions
  • Journal Announcements

Authors

  • ASM Author Center
  • Submit a Manuscript
  • Article Types
  • Ethics
  • Contact Us

Follow #AppEnvMicro

@ASMicrobiology

       

ASM Journals

ASM journals are the most prominent publications in the field, delivering up-to-date and authoritative coverage of both basic and clinical microbiology.

About ASM | Contact Us | Press Room

 

ASM is a member of

Scientific Society Publisher Alliance

 

American Society for Microbiology
1752 N St. NW
Washington, DC 20036
Phone: (202) 737-3600

Copyright © 2021 American Society for Microbiology | Privacy Policy | Website feedback

 

Print ISSN: 0099-2240; Online ISSN: 1098-5336