Author Topic: Quality of academic papers  (Read 1659 times)

0 Members and 1 Guest are viewing this topic.

Offline CirclotronTopic starter

  • Super Contributor
  • ***
  • Posts: 3168
  • Country: au
Quality of academic papers
« on: December 06, 2022, 05:52:20 am »
I sometimes wonder about academic papers.
Can anybody see the problem with this graph?

 

Offline armandine2

  • Frequent Contributor
  • **
  • Posts: 589
  • Country: gb
Re: Quality of academic papers
« Reply #1 on: December 06, 2022, 07:49:18 am »
not a big fan of the choice of x-axis scale numbers.

I have recently redrawn a figure from an electronics "textbook", with only partial success :palm:

Edited to add observation: three of the lines in the key don't represent any lines on the graph
« Last Edit: December 06, 2022, 08:08:12 am by armandine2 »
Funny, the things you have the hardest time parting with are the things you need the least - Bob Dylan
 

Offline CirclotronTopic starter

  • Super Contributor
  • ***
  • Posts: 3168
  • Country: au
Re: Quality of academic papers
« Reply #2 on: December 06, 2022, 09:34:43 am »
My point was that the legend does not match the graph lines.
 

Offline Alex Eisenhut

  • Super Contributor
  • ***
  • Posts: 3330
  • Country: ca
  • Place text here.
Re: Quality of academic papers
« Reply #3 on: December 06, 2022, 02:17:23 pm »
My point was that the legend does not match the graph lines.
Not only that but the graph gives me that optical illusion flashing points effect.
Hoarder of 8-bit Commodore relics and 1960s Tektronix 500-series stuff. Unconventional interior decorator.
 

Offline jonpaul

  • Super Contributor
  • ***
  • Posts: 3322
  • Country: fr
Re: Quality of academic papers
« Reply #4 on: December 06, 2022, 02:31:24 pm »
I used to enjoy the AES, SMPTE, IEEE paper's 

Last decade the papers fall into two horrible categories 

1. student ( usually from Asia) need to have a published paper
Useless research poorly explained and expressed 

2. thinly veiled product advertising or announcement disguised as an academic paper.

Just the ramblings of an old retired EE 

Jon
Jean-Paul  the Internet Dinosaur
 
The following users thanked this post: RJSV

Offline T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 21609
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: Quality of academic papers
« Reply #5 on: December 06, 2022, 04:52:15 pm »
Was that in a reputable journal or what?

There are, of course, the families of predatory journals, often named adjacent to reputable ones, and which exist in part to extract value from naive or underperforming academics; serve as advertising; one can say "it's published research!!"; etc.

Knowledge isn't some static thing, it is an ongoing process; one must never let their guard down, always be critical of who, what, where, etc.  Knowledge decays over time, not just because our brains are made of meat, but because the circumstances of those facts also changes over time.  Physical laws maybe not so much, but we hold a hell of a lot more facts about human relationships, and things we've created, than we do immutable laws of the universe.  Facts like interpersonal relationships, literature and mythology, the myriad things we've created and interconnected, etc.

Meanwhile, the circle of reputable journals contains, and is well cited by, effective and insightful research (or at least, it is from time to time, depending on how insightful you might consider the more abundant but incremental sort of reports; but that's still technically scientific progress).  The circle of irreputable journals might be well cited among themselves*, but not by the other group.

*I don't know that I've heard about this particular aspect before, come to think of it.  Further reading is suggested. :)

You see the same patterns of relationships and information flows in public media, as misinformation is created and spread among conservative and lazy publishers, and only belatedly corrected by those that care to.  Being aware of these patterns is a mandatory part of critical thinking, especially these days.  Unfortunately, especially these days, it is also a big tax having to pore over so much material to find where it comes from and what it's really saying.  (Case in point, see China's recent explosion of misinformation on Twitter.)

[I would normally refrain from calling out sources such as "conservative" or "China", but in these cases there is objective research on the matter, so it feels appropriate to call them out as such.]

So, it is ultimately up to the individual to collect and synthesize information into a web of trust; especially these days with our traditional trust networks under attack.  There will always and necessarily be an undercurrent of poor quality or factually-wrong information, and we must all work to push it away, while better collecting, curating and cultivating the good stuff.

Tim
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 

Online RoGeorge

  • Super Contributor
  • ***
  • Posts: 6147
  • Country: ro
Re: Quality of academic papers
« Reply #6 on: December 06, 2022, 05:12:50 pm »
The weirdest I've seen was the CERN "paper":  Physics paper sets record with more than 5,000 authors
https://www.nature.com/articles/nature.2015.17567

Then, there are the clickbait-like titled papers, e.g. "Attention is all you need" (a paper about artificial intelligence) and so on.  The most unbearable are those speaking in a personal tone, others in a casual manner, some are even trying to be funny or to allude to memes.

It's getting wilder by the day.  ;D
« Last Edit: December 06, 2022, 08:41:28 pm by RoGeorge »
 

Online Kleinstein

  • Super Contributor
  • ***
  • Posts: 14080
  • Country: de
Re: Quality of academic papers
« Reply #7 on: December 06, 2022, 07:00:05 pm »
There is no strict separation in good and bad journals. Even the normally considered better one have quite some rather poor quality articles. A problem quite common with US autors is that essentially the same article is repeatedly published in several journals (likely also multiple times in the same yournal) over a few years. Sometimes there is at least a little incremental advance in research, but often it is hard to find.

The US system of judging by quantety of publications is a problem here.
The number of publications has just expolded and that alone makes it hard to find the needle in the hay-stack. The number of good articles is hardly going up.

The peer review system is also working with different standards. Especially with conference papers the standards are often lower  - with some justification.
However some journals seem to not take it too serious too - though ideally the journal has not that much influence (maybe make it easier to give it a 2nd chance with a new reviewer).

Those rediculous long list of authors are a problem too: even with just 3 it is hard to make all agree on a text and usually 1 or 2 do all the work and the 3rd is just the boss.
 

Offline TimFox

  • Super Contributor
  • ***
  • Posts: 7934
  • Country: us
  • Retired, now restoring antique test equipment
Re: Quality of academic papers
« Reply #8 on: December 06, 2022, 08:35:23 pm »
Personally, I used "Grapher" from Golden Software to create technical graphs before I retired.

I liked the results so much that I purchased an expensive set for use on personal projects after I retired.

It may not keep me from making that dumb mistake you showed, but the text comes out looking "literate", and I can change formats to look like proper graphics.

A very simple example, using my choice on formats, is appended as a jpeg.
Since this was intended for insertion into a Word document, it is monochrome.



1656568-0
 
The following users thanked this post: RJSV

Offline tooki

  • Super Contributor
  • ***
  • Posts: 11342
  • Country: ch
Re: Quality of academic papers
« Reply #9 on: December 06, 2022, 09:25:26 pm »
The weirdest I've seen was the CERN "paper":  Physics paper sets record with more than 5,000 authors
https://www.nature.com/articles/nature.2015.17567
I used to work for a company that makes reference management software (competitor of e.g. Endnote and Zotero), and they used those record-breaking CERN papers as acid tests to make sure the code would properly handle absurd numbers of authors… 
 

Offline CatalinaWOW

  • Super Contributor
  • ***
  • Posts: 5173
  • Country: us
Re: Quality of academic papers
« Reply #10 on: December 06, 2022, 09:40:00 pm »
Multiple completely different subjects here.  The original graph appears to contain useful data, poorly presented.  While it doesn't appear to be an earthshaking contribution to knowledge, I can't say whether it is one of a family of nearly identical papers generated to enhance pub count.

There is also an element of snobbery at places in this thread.  Though I wouldn't want to see a performance characterization of a particular brand and model number component being accepted as someone's PhD thesis I certainly find such data useful and want to see it published.
 

Offline RJSV

  • Super Contributor
  • ***
  • Posts: 2102
  • Country: us
Re: Quality of academic papers
« Reply #11 on: December 06, 2022, 10:17:42 pm »
   Don't see it, but many folks involved in complexity overlook mistakes, until later or until a really not distracted time.  What's the defect ?

   And, Alex E.; I see a bit of 'crawling' dots on the green dots...The green shade is most efficient for human sensitivity.  Those green dots wiggle slightly, when head moves.
 

Offline IanB

  • Super Contributor
  • ***
  • Posts: 11790
  • Country: us
Re: Quality of academic papers
« Reply #12 on: December 06, 2022, 10:24:54 pm »
A very simple example, using my choice on formats, is appended as a jpeg.
Since this was intended for insertion into a Word document, it is monochrome.

That does look good.

By the by, I tend to use color freely in Word documents these days. For one, because they are mostly read on a screen and rarely printed anymore, and secondly if someone does want to print it then color printers are very common.
 

Offline Zero999

  • Super Contributor
  • ***
  • Posts: 19345
  • Country: gb
  • 0999
Re: Quality of academic papers
« Reply #13 on: December 06, 2022, 11:00:12 pm »
Veiled advertising is a big problem. It's important to check who funded the study to find any potential conflicts of interest.

Political motives i.e. social justice as also a problem. It used to mostly affect the humanities but has started creeping into medicine. I've not seen it in engineering yet. This is often a bit more tricky to spot, as more often than not the author is doing it for their own cause, rather than a certain party.
 

Offline TimFox

  • Super Contributor
  • ***
  • Posts: 7934
  • Country: us
  • Retired, now restoring antique test equipment
Re: Quality of academic papers
« Reply #14 on: December 06, 2022, 11:02:11 pm »
Especially when reading on a screen, using different colors for each curve is certainly a good idea.
Before I retired, we were asked to keep color printing to a minimum in-house, for cost-control reasons, and I only have a mono printer at home.
What I like about this program is that they have a good selection of the usual fonts, with bold-face optional.
Some other programs, when using log scales, force one to have integer powers of 10 as maximum and minimum, but Grapher allows arbitrary end values, with choice of "E" or "10N" notation.
It also allows one to choose major and minor tick increments, and to show limited numbers of major tick labels, if desired.
Besides colors, it allows choosing line widths and dashes, and using multiple axes (one pair for each curve).
One thing I don't like:  it uses Excel as the input for values, which must be in parallel vertical columns.
However, if I see an error on the graph, and go back to the Excel file to correct that entry, I need to re-name the Excel file and select it in Grapher;  otherwise, it doesn't recognize the change from the original data.
It's not cheap:  a single-user "perpetual" purchase, with 1 year support is $679;  a single-user subscription is $329/year.
https://www.goldensoftware.com/products/grapher
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14309
  • Country: fr
Re: Quality of academic papers
« Reply #15 on: December 06, 2022, 11:13:35 pm »
My point was that the legend does not match the graph lines.

Yep. There's a few other points that look sub-par on this graph, but the legend is certainly the worst.
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14309
  • Country: fr
Re: Quality of academic papers
« Reply #16 on: December 06, 2022, 11:16:58 pm »
Personally, I used "Grapher" from Golden Software to create technical graphs before I retired.

Seems good. For those not afraid of command-line tools (and those that can actually benefit from that for automating graph generation for instance), GnuPlot is great. There's a learning curve though to get exactly what you want, but they provide a shitload of examples.

Otherwise, SciDavis isn't bad and is GUI-based.

Both can export in a very large number of formats including of course vector graphics.
 

Offline Someone

  • Super Contributor
  • ***
  • Posts: 4510
  • Country: au
    • send complaints here
Re: Quality of academic papers
« Reply #17 on: December 06, 2022, 11:19:29 pm »
A very simple example, using my choice on formats, is appended as a jpeg.
Since this was intended for insertion into a Word document, it is monochrome.
That does look good.

By the by, I tend to use color freely in Word documents these days. For one, because they are mostly read on a screen and rarely printed anymore, and secondly if someone does want to print it then color printers are very common.
There are alternative color sequences/distributions/scales that retain some/most of the intent when viewed without color (B&W reproduction, or color blind viewer):
https://bids.github.io/colormap/

It's not cheap:  a single-user "perpetual" purchase, with 1 year support is $679;  a single-user subscription is $329/year.
https://www.goldensoftware.com/products/grapher
Blimey! Glad there are open source alternatives these days:
https://matplotlib.org
and gnuplot as mentioned above by SiliconWizard:
http://www.gnuplot.info

 

Offline CirclotronTopic starter

  • Super Contributor
  • ***
  • Posts: 3168
  • Country: au
Re: Quality of academic papers
« Reply #18 on: December 07, 2022, 12:40:24 am »
 

Offline T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 21609
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: Quality of academic papers
« Reply #19 on: December 07, 2022, 03:29:58 am »
Yeh, not top-tier I guess.  Apparently they've retracted a bunch of articles for lack of peer review or other shady circumstances.  But like mentioned about conference papers and stuff, maybe that's a kind of thing that's mildly reviewed and kinda not worth [reviewing heavily], *shrug*.

Tim
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 

Online hans

  • Super Contributor
  • ***
  • Posts: 1626
  • Country: nl
Re: Quality of academic papers
« Reply #20 on: December 07, 2022, 04:24:11 pm »
The scientific community relies on redundancy to work. If there is only one paper claiming to have made some important discovery, then it virtually doesn't exist. If that discovery is then cited, re-tested, re-used, re-confirmed.. then you can start to build a pattern and knowledge.

I'm not amazed by these kinds of styling errors. Yes they should be caught, especially in a "journal" where reviewing should be a lot more strict than just a small conference.. but it does happen. I've seen some very big papers in my field with hundreds of citations at ACM/IEEE, but are written quite poorly in terms of spelling, punctuality, level of detail or structure.

Stuff like a graph displaying a handful of phasor diagrams with a dozen arrows and no labels. With the caption and article text saying: "Fig X. contains several phasor diagrams of the system in different circumstances, showing the various factors that may affect it". Well great, that tells me exactly nothing, except for if I didn't know what a phasor diagram is so I could look up these weird arrows.

Or a paper that tried to squeeze separate 2 designs into one article which were assembled together like a classic TV series, where they switch between scenes A and B to keep you sucked in to watch the whole thing. When I finally had sifted through both contributions, I concluded they could have been published independently, so why did I have to read through 20+ pages of 2-column text (with figures referenced 3 pages back and ahead) again to confuse myself?

Etc. I can probably rant for days. And to be fair, probably other academics will go off on my papers as well for their own reasons. It is what it is. Now if papers don't undermine anything on the quality and reproducibility of the research, then it's all fine. I think I would pattern-match these lines as if the graph was B/W thus on shape, which seems to make sense that a larger energy mixture corresponds to a faster developing wavefront.
It's not cheap:  a single-user "perpetual" purchase, with 1 year support is $679;  a single-user subscription is $329/year.
https://www.goldensoftware.com/products/grapher
Blimey! Glad there are open source alternatives these days:
https://matplotlib.org
and gnuplot as mentioned above by SiliconWizard:
http://www.gnuplot.info

Or my favorite: pgfplot package for Tikz in LaTeX.

Yes LaTeX is very fiddly and time consuming to use, but it does make very pretty documents and figures. And I'm always very happy with the level of control I've on my figures (like procedurally generate), and the ease of which I can keep them consistent between paperwork and presentation slides (also made in LaTeX), where I can put slide animations within my graphs or drawings if needed.
« Last Edit: December 07, 2022, 04:30:00 pm by hans »
 
The following users thanked this post: Someone

Offline TimFox

  • Super Contributor
  • ***
  • Posts: 7934
  • Country: us
  • Retired, now restoring antique test equipment
Re: Quality of academic papers
« Reply #21 on: December 07, 2022, 10:29:54 pm »
To be fair about the price, Grapher from Golden Software has far more functionality then I need, but I learned to like it.
 
The following users thanked this post: Someone

Offline coppercone2

  • Super Contributor
  • ***
  • Posts: 9247
  • Country: us
  • $
Re: Quality of academic papers
« Reply #22 on: December 07, 2022, 10:59:52 pm »
there is also a problem with fake papers that have fake data or doctored data etc. journals want the numbers for being bigger then other journals so its loosely enforced IMO

« Last Edit: December 07, 2022, 11:01:39 pm by coppercone2 »
 

Offline Someone

  • Super Contributor
  • ***
  • Posts: 4510
  • Country: au
    • send complaints here
Re: Quality of academic papers
« Reply #23 on: December 08, 2022, 02:46:16 am »
To be fair about the price, Grapher from Golden Software has far more functionality then I need, but I learned to like it.
For sure, plenty of people have bought various expensive packages (Matlab, or statistics platforms) just for the high quality and mouse driven plotting.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf