The paper studies how three selected LLMs, Llama-2-chat 7B, Mistral-7B-Instruct and GPT-3.5-turbo, write counter-arguments for 13,504 sociopolitical posts from the pre-2016 r/ChangeMyView corpus. Each model receives only the post plus the instruction 'You have one chance to change my view'; one response is sampled at temperature 0.9, top_p 0.6 and a 600-token cap. The analysis does not observe intentions directly. It reuses nine LSTM classifiers that assign social dimensions such as knowledge, trust, support, similarity and conflict, represents a message by its maximum sentence score, binarizes at the 85th percentile and applies a length discount. Under this operationalization, all three generated corpora almost always contain language classified as trust, with odds ratios from 375.31 to 611.28 relative to human delta comments. Generated messages also contain more dimensions per message but less knowledge and similarity, the two dimensions most associated with delta in the human data. The trust result appears to be driven by recurring formulas such as 'I understand your perspective' and does not demonstrate felt trust or Habermasian intention. To examine interaction dynamics, the paper compares dimensions in a post and its comment. Some human reciprocity patterns are associated with receiving a delta, and the models reproduce part of those patterns; GPT-3.5 shows stronger reciprocity on several dimensions. This comparison is observational and conditional: it does not show that any generated response changed an opinion. In an additional exploratory study, 40 US Prolific workers evaluate 100 triples containing a post, one human delta comment and one GPT-3.5 response. By majority vote, workers select the GPT response as more likely to change the original poster's view in 83 of 100 cases, with Krippendorff's alpha of 0.79, and report agreement with 92 GPT comments versus 66 human comments. The raters are not the original posters, however, and judge hypothetical third-party persuasiveness; there is no pre/post measure or observed opinion change. Finally, the link presented as 'our code & data' contains only a generic .gitignore and a CC0 license, so the filtering, generations, classifier labels, statistics and survey cannot be reproduced from the audited public artifact.
Research question
What social dimensions do automatic classifiers detect in LLM counterarguments, how do they differ from human comments with and without delta, to what extent do they reproduce post-comment dynamics associated with opinion change, and which of a human response with delta or a GPT-3.5 response do third parties judge as more persuasive?