Data Analysis
This paper utilizes the Grounded Theory approach to conduct a content
analysis of sermons generated by ChatGPT, following the model proposed
by Charmaz27. This methodology has been adopted for
the study of homiletics by scholars such as
Pieterse28. Extensive explanations of Grounded theory
and its place in research can be found in common literature and would
not be expounded on within this paper. This paper would be following the
methodology outlined by Pieterse, following a constructivist
epistemology where the analyst and authors of texts are co-constructing
the categories and concepts29. Grounded Theory is
particularly suitable for studying the content of sermons as it enables
the analysis of concept relationships within a Christian social and
cultural context. Chramaz directs the researcher to examine processes
and make the study of action the central focus of research. In relation
to this paper, the actions of the preacher during sermon delivery are
the primary focus. Thus, during the initial coding phase, codes were
formulated as “Preacher claims… Preacher asserts…
Preacher invites…etc.”
Before coding commenced, a review of current literature on preaching,
the Gospel message, and evangelism was conducted to develop sensitizing
concepts. According to Charmaz, through familiarization with literature
in the area of study being conducted, the researcher would be able to
develop sensitizing concepts which she defines as “background ideas
that inform the overall research problem”30. This
literature review highlighted the issue of Sin, the substitutionary
atonement of Jesus Christ, eternal life, and reconciliation with God as
prominent ideas. The idea of preaching being an event to impact people
to Christ also arose. These concepts informed the data analysis
throughout the study.
In the initial coding phase of the first set of sermon texts, the sermon
was first segmented based on the preacher’s communicative focus. These
segments were then further coded according to the actions of the
preacher within each segment using Microsoft Word and Excel. This
reflexive process involved constant comparison of codes and segments to
ensure consistency across all sermons. The initial coding process was
repeated until every sermon was analyzed. The codes were subsequently
organized into categories through continuous comparison of codes and
segments. Certain codes were selected based on their overriding
significance. During the initial coding phase, it became apparent that
the sermons indicated the preacher’s personal witness to the
transformative power of the gospel in people’s lives, but its
relationship to other categories remained unclear and required further
investigation. Consequently, sermons incorporating personal
illustrations of the preacher were generated to explore this aspect
further.
During the focused coding phase, the second set of sermons was first
coded line-by-line. These codes were then compared to the existing
initial categories from the first set of sermon texts. The initial
categories were then further enriched, and the properties and dimensions
of the existing categories were specified. Memos were written and
analyzed to help enrich the core category. This was followed by a final
stage of theoretical coding, where advanced memos were compared, sorted,
and categories reviewed. During this phase, the necessity of the
transformative power of the gospel was identified as the central
category. Associations between themes were drawn based on the data and
initial literature review. At this stage, an additional five sermons
were generated to examine the structure and content of an evangelistic
sermon with minimal prompts. As no new categories emerged during the
coding process, it was determined that ”theoretical saturation” had been
achieved.
Each sermon was then examined for the presence of the five key elements
that constitute the gospel call and their occurrence in each sermon was
quantified to assess the effectiveness of ChatGPT in generating
theologically comprehensive evangelistic sermons. Every Bible scripture
quoted within the generated sermon texts was scrutinized to ensure
verbatim quotations from the Bible using the Google Search engine and
available online Bible translations. Finally, the structure of the
sermons was compared to existing literature.