Data Analysis

This paper utilizes the Grounded Theory approach to conduct a content analysis of sermons generated by ChatGPT, following the model proposed by Charmaz27. This methodology has been adopted for the study of homiletics by scholars such as Pieterse28. Extensive explanations of Grounded theory and its place in research can be found in common literature and would not be expounded on within this paper. This paper would be following the methodology outlined by Pieterse, following a constructivist epistemology where the analyst and authors of texts are co-constructing the categories and concepts29. Grounded Theory is particularly suitable for studying the content of sermons as it enables the analysis of concept relationships within a Christian social and cultural context. Chramaz directs the researcher to examine processes and make the study of action the central focus of research. In relation to this paper, the actions of the preacher during sermon delivery are the primary focus. Thus, during the initial coding phase, codes were formulated as “Preacher claims… Preacher asserts… Preacher invites…etc.”
Before coding commenced, a review of current literature on preaching, the Gospel message, and evangelism was conducted to develop sensitizing concepts. According to Charmaz, through familiarization with literature in the area of study being conducted, the researcher would be able to develop sensitizing concepts which she defines as “background ideas that inform the overall research problem”30. This literature review highlighted the issue of Sin, the substitutionary atonement of Jesus Christ, eternal life, and reconciliation with God as prominent ideas. The idea of preaching being an event to impact people to Christ also arose. These concepts informed the data analysis throughout the study.
In the initial coding phase of the first set of sermon texts, the sermon was first segmented based on the preacher’s communicative focus. These segments were then further coded according to the actions of the preacher within each segment using Microsoft Word and Excel. This reflexive process involved constant comparison of codes and segments to ensure consistency across all sermons. The initial coding process was repeated until every sermon was analyzed. The codes were subsequently organized into categories through continuous comparison of codes and segments. Certain codes were selected based on their overriding significance. During the initial coding phase, it became apparent that the sermons indicated the preacher’s personal witness to the transformative power of the gospel in people’s lives, but its relationship to other categories remained unclear and required further investigation. Consequently, sermons incorporating personal illustrations of the preacher were generated to explore this aspect further.
During the focused coding phase, the second set of sermons was first coded line-by-line. These codes were then compared to the existing initial categories from the first set of sermon texts. The initial categories were then further enriched, and the properties and dimensions of the existing categories were specified. Memos were written and analyzed to help enrich the core category. This was followed by a final stage of theoretical coding, where advanced memos were compared, sorted, and categories reviewed. During this phase, the necessity of the transformative power of the gospel was identified as the central category. Associations between themes were drawn based on the data and initial literature review. At this stage, an additional five sermons were generated to examine the structure and content of an evangelistic sermon with minimal prompts. As no new categories emerged during the coding process, it was determined that ”theoretical saturation” had been achieved.
Each sermon was then examined for the presence of the five key elements that constitute the gospel call and their occurrence in each sermon was quantified to assess the effectiveness of ChatGPT in generating theologically comprehensive evangelistic sermons. Every Bible scripture quoted within the generated sermon texts was scrutinized to ensure verbatim quotations from the Bible using the Google Search engine and available online Bible translations. Finally, the structure of the sermons was compared to existing literature.